AUGUST 1961 


Journal of Educational 
Psychology 


CONTENTS 
An rey of the Factorial Dimensions of the Achievement Motiva- 


JAMES V. MITCHELL, JR. 
in Deaf and Hearing 


poor L. KATES, WILLIAM W. KATES, JAMES MICHAEL, AND 
TERRENCE M. WALSH 


Cross-Validation Groups, Extreme Groups, and the Prediction wi 
Academic Achievement 
LEONARD J. KAZMIER 


A Note on Relationships between Student and Parent Minnesota 
Teacher Attitude Inventory Scores 
ROBERT E. CAMPBELL AND JOHN E. HORROCKS 


Test Anxiety and the Intellectual Performance of College Students.... 201 
IRWIN G. SARASON 


Effects of Memory, Evaluative, and Creative Reading Sets on Test 
E. PAUL TORRANCE AND JUDSON A. HARMON 


The Effects of Visual Discrimination Pretraining with Word and 
Letter Stimuli on Learning to Read a Word List in Kindergarten 


AMERICAN PSYCHOLOGICAL ASSOCIATION 


4 bd ° vay 
| 
Vol. 52 No. 4 
Pa i 
it 
\ 
4 
4 A 
| 
179 
4 
188 
| 
199 
| 
SIEGMAR MUEHL 
‘ 


Journal of Educational Psychology 


RayMonpD G. KUHLEN, Editor 
Syracuse University 


Advisory Editors 


Joun B. CaRROLL Carson McGuire 

Harvard University University of Texas 
Lee J. CRONBACH H. Mussen 

University of Illinois University of California, Berkeley 
Francis J. DiVesta T. Ernest NEWLAND 

Syracuse University University of Illinois 
Cuester W. Harris H. H. RemMers 

University of Wisconsin Purdue University 
Ernest R. HILGARD Juwtan C. STANLEY 

Stanford University University of Wisconsin 
Hosss J. B. Stroup 

George Peabody College for Teachers State University of Iowa 
DorotrHeA McCartTuy Donan E. SuPer 

Fordham University Teachers College, Columbia University 


Grorce G. THOMPSON 
Ohio State University 


The Journal of Educational Psychology publishes original investigations of problems of 
learning, teaching, and the psychological development of the individual. It contains articles 
on the following subjects: the psychology of school subjects; experimental studies of learning; 
the development of interests, attitudes, personality, social relations, and vocational orienta- 
tion, particularly (but not exclusively) as related to school adjustment or the impact of the 
school program upon development and adjustment; emotion, motivation, and character; 
mental development; and methods. The last includes tests, statistical techniques, and re- 
search techniques in cross-sectional and developmental studies. Studies published deal with 
all levels of education—preschool, elementary, secondary, college, and adult education— 
and with all age groups. 

Manuscripts and correspondence on editorial matters should be addressed to the Editor: 
Raymond é. Kuhlen, Department of Psychology, Syracuse University, 123 College Place, 
Syracuse 10, New York. 

All manuscripts should be submitted im duplicate to facilitate referral to advisory editors 
and must be prepared to conform to the style and procedures described in the Publication 
Manual of the American Fsychological Association. Because of the great demand for journal 
space, authors should give special attention to reducing the length of manuscripts while 
maintaining clarity of expression. Ordinarily, papers should not exceed 15 typed pages. 
Accepted manuscripts are published in order of their receipt. An author may, however, 
request early publication of accepted manuscripts if he is willing to assume the cost of print- 


ing the necessary extra pages. 


Artaur C. HorrmMan HELEN OrR 
Managing Editor Promotion Manager 


Communications including subscriptions, orders of back issues, and changes of address 
should be addressed to the American Psychological Association, 1333 Sixteenth Street, N.W., 
Washington 6, D. C. Address changes must reach the APA by the 10th of the month to take 
effect the following month. Undelivered copies resulting from address changes will not be 
replaced; subscribers should notify the post office that they will guarantee second-class for- 
— postage. Other claims for undelivered copies must be made within four months of 
publication. 

Annual subscription: $10.00 (Foreign $10.50). Single copies, $2.00. 


PUBLISHED BIMONTHLY BY: 


THE AMERICAN PSYCHOLOGICAL ASSOCIATION, INC. 
Mt. Royal & Guilford Aves., Baltimore 2, Md. 
and 1333 Sixteenth Street N.W., Washington 6, D.C. 
Second-Class postage paid at Baltimore, Md. 
Copyright, 1961 by the American Psychologica! Association, Inc. 


iy 
a 
4 
id 
at 
Sag 
| 
wal 
’ 
4 a 
ab 
‘ 
; 
ef 
= 


JOURNAL OF 
EDUCATIONAL PSYCHOLOGY 


Volume 52 August 1961 Number 4 


— — 


AN ANALYSIS OF THE FACTORIAL DIMENSIONS OF 
THE ACHIEVEMENT MOTIVATION CONSTRUCT 


JAMES V. MITCHELL, Jr. 
University of Texas 


The construct of achievement mo- 
tivation has been the object of con- 
siderable research and discussion in 
recent years (Atkinson, 1958; McClel- 
land, Atkinson, Clark, & Lowell, 
1953), and it is only too apparent that 
there are wide differences of opinion 
about the adequacy of current efforts 
to measure that intriguing variable 
(Atkinson, 1960; Blake & Mouton, 
1959). Much of that effort, of course, 
has been focused on the assessment of 
n Achievement by means of analy- 
ses of TAT protocols, a technique de- 
veloped by McClelland et al. (1953) 
and now regarded as sui generis by its 
devotees. One adherent of this tech- 
nique has even relegated all other 
methods of assessing achievement mo- 
tivation to the limbo of “ad hoe” de- 
vices, and he admonishes all users of 
such devices not to assume that the 
theo :etical conceptions and factual in- 
formation resulting from projective 
analyses of n Achievement are equally 
pertinent to studies involving the “ad 
hoe” assessment devices (Atkinson, 
1960, pp. 275-276) . Others not quite so 
identified with the McClelland ap- 
proach seem equally insistent that n 
Achievement scores from different in- 
struments exhibit significant relation- 
ships to one another and to the com- 
mon behavioral data they are supposed 
to measure. With discouraging regu- 
larity, however, investigators have re- 
ported the absence of relationship 


between different measures of achieve- 
ment motivation (Bendig, 1957; Hi- 
melstein, Eschenbach, & Carp, 1958; 
Melikian, 1958), and attempts to pre- 
dict actual behavioral criteria have 
often resulted in essentially negative 
findings, confusion, or vast conceptual 
reinterpretation (Birney, 1958; Laza- 
rus, Backer, Broverman, & Mayer, 
1957; Vogel, Baker, & Lazarus, 1958). 
It is readily apparent that there is a 
rather pronounced lack of consensus 
about both the conceptual and meth- 
odological aspects of the achievement 
motivation construct, and there is an 
urgent need for additional research 
that will help to provide structure for 
its currently amorphous state of ex- 
istence. 

The present investigation came 
about as a result of a larger study of 
the cultural and situational determi- 
nants of achievement motivation for 
a group of college students. For this 
larger study the measurement of 
achievement motivation was obviously 
a critical operation, and since the re- 
search literature provided ample evi- 
dence of the difficulties involved in this 
kind of assessment, it seemed impera- 
tive to select as many different meas- 
ures of achievement motivation as 
would be necessary to assure a thor- 
ough representation of that domain 
and then to make a careful analysis 
of their correlates and factorial dimen- 
sions. It was early recognized that such 
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a procedure should result in findings 
that would have general theoretical 
significance and applicability, because 
they would help to shed light on 
whether achievement motivation was 
a unitary construct with invariable 
meaning, as many seem to regard it, 
or whether it referred in fact to sev- 
eral psychological dimensions, each 
with different implications and differ- 
ent empirical correlates. It is this task 
that defines the purpose of the present 
investigation. 


PROCEDURE 


The following instruments were employed 
in the analysis to represent the domain of 
possible achievement related attitudes. 

1. McClelland Test of Achievement Mo- 
tivation. This instrument was administered 
and scored in accordance with the instruc- 
tions given by McClelland in his book The 
Achievement Motive (McClelland et al., 
1953). In scoring a representative sample of 
protocols provided in that book, the present 
investigator established that his scores cor- 
related 92 with those reported by the test 
author. 

2. A sentence completion test of achieve- 
ment motivation. This test was developed 
by the investigator as a simplified projective 
measure of achievement motivation, with 
the thought that such a measure might well 
have more validity than an instrument with 
the scoring complexity of the McClelland 
test. The instrument consisted of 46 items, 
some developed by the author and some 
taken from experimental tests developed by 
other investigators. The items selected rep- 
resented various degrees of achievement 
orientation, from strongly achievement 
oriented to complete lack of achievement 
orientation. The instrument was scored by 
applying to each item the very first cri- 
terion of the McClelland system for scoring 
the TAT protocols, viz., the presence or 
absence of an “achievement goal.” Accord- 
ing to McClelland, the existence of an 
achievement goal is indicated by any state- 
ment which implies “success in competition 
with some standard of excellence” (McClel- 
land et al., 1953, p. 110). If such were ex- 
pressed or implied in the completion of any 
item, that item was scored +1. If the item 
completion involved no achievement im- 
agery of this type, it was scored 0. If the 
item completion expressed or implied an 
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evident lack of success or interest in com- 
peting with some standard of excellence, it 
was scored —1. The score on the test was 
simply the algebraic sum of the item scores. 

3. An adjective checklist containing 
achievement related items. A few achieve- 
ment related adjectives were added to those 
similarly oriented and already present in 
the Bills’ Index of Adjustment and Values 
(Bills, Vance, & McLean, 1951), and for 
each of these adjectives the subject rated 
himself on a 5-point scale. The subject's 
achievement motivation score on this in- 
strument was the total of the numerical rat- 
ings for the contributing adjectives. Cor- 
rected split-half reliability for this score was 
34. Remaining items not contributing to 
this score served to mask the real purpose 
for which the instrument was being ad- 
ministered. Noncontributing items also 
served this same function for the two fol- 
lowing instruments. 

4. An inventory-type test of achievement 
motivation. This inventory contained items 
of the True-False variety, many of which 
were borrowed from the MMPI. Other 
items related to achievement motivation 
were added by the investigator. The cor- 
rected split-half reliability of this instru- 
ment was .75. 

5. A multiple-choice questionnaire in- 
cluding achievement motivation items. The 
alternatives for each item in the question- 
naire were serially ordered so as to define 
choices along a continuum representing 
greater of lesser degrees of the variable in 
question. The number of an alternative 
chosen thus defined a position on the con- 
tinuum and the individual’s score for that 
particular item. The corrected split-half re- 
liability of this instrument was 83. 

6. Taylor Manifest Anxiety scale (1953). 
This scale was included in the battery to 
clarify whether the imputed drive proper- 
ties of manifest anxiety, as hypothesized by 
the disciples of Hull, would receive de facto 
verification from the factor analysis of these 
achievement motivation variables. 

7. Level of aspiration measures. All sub- 
jects were members of the investigator’s 
sections in elementary educational psychol- 
ogy. Before the first and last examinations 
of the semester all subjects were asked to 
record their levels of aspiration by answer- 
ing the question: “What grade can you 
reasonably hope to attain on this examina- 
tion?” These aspiration estimates were origi- 
nally received in letter grade form and then 
later converted to a 10-point numerical scale 
which recognized + and — distinctions, eg., 


| 
4 
| 
| 
| 
| 
: | 
ag 
J 
ok 
‘4 
fe 


FACTORIAL DIMENSIONS OF ACHIEVEMENT MOTIVATION 181 


C+, B-—, etc. Actual grade received on the 
examination was also converted to the same 
10-point numerical system. For each sub- 
ject the numerical value of the actual grade 
received was subtracted from the numerical 
value of his level of aspiration estimate. 
This was done for both the first and last 
examinations, resulting in two indices which 
represented the extent to which the subject 
actually attained or approached his level of 
aspiration. 

8. A measure of discrepancy between 
achievement and potentiality. The numeri- 
cal grade averages for all subjects for the 
semester in which the study was conducted 
were computed, normalized, and converted 
into standard score form. The ACE scores 
for all subjects were similarly normalized, 
and converted into standard score form. A 
discrepancy score was computed for each 
subject by subtracting his standard score 
for the ACE from his standard score for his 
semester grade point average (GPA — 
ACE). 

All the afore-mentioned data were se- 
cured from 131 female college students who 
were in a teacher training curriculum. The 
analysis was divided into two parts: (a) 
The scores from the various instruments 
were correlated with one another and with 
grade point average criteria, and (b) as a re- 
sult of the insights gained from the first 
analysis, all of the individual items com- 
prising the various instruments, some 29 
variables in all, were then intercorrelated, 
subjected to a centroid factor analysis as 
described by Thurstone (1947), and subse- 
quently rotated to orthogonal simple struc- 
ture by means of Kaiser’s varimax technique 
(1958). 


RESULTS AND DIscussION 


Table 1 shows the intercorrelations 
between measures of achievement mo- 
tivation, ACE, and GPA. The GPA— 
ACE discrepancy score (last column) 
here functions as the criterion variable 
for achievement motivation. Immedi- 
ately apparent are the differences in 
correlations between this criterion and 
the various measures of achievement 
motivation. The simple adjective 
checklist was the best predictor of 
the criterion. The achievement moti- 
vation inventory was significantly re- 
lated to the criterion, but, unlike the 


adjective checklist, had a significant 
negative correlation with ACE as well. 
The level of aspiration measures were 
related significantly to the criterion, 
but this may be partly artifactual in 
nature, resulting from the presence of 
a common component, grades, in the 
make-up of predictor and criterion. 
Manifest anxiety was quite unrelated 
to the criterion. Neither of the projec- 
tive instruments (sentence completion 
and TAT n Achievement) was related 
to the criterion. Interestingly enough, 
however, the sentence completion test 
did have a significant and moderately 
high negative correlation with ACE 
and a lower but still significant nega- 
tive correlation with GPA. These cor- 
relations, along with the negative cor- 
relation reported earlier for the 
achievement motivation inventory and 
ACE, re-emphasize the necessity for 
cautious empirical inquiry into the 
real dimensions and correlates of com- 
plex measures of complex constructs 
like achievement motivation. 

The intercorrelations between the 
various measures of achievement mo- 
tivation included in Table 1 show some 
interesting and often perplexing 
trends. The adjective checklist is sig- 
nificantly correlated with the achieve- 
ment motivation inventory but not 
with the sentence completion test. The 
sentence completion test is, however, 
significantly correlated with the 
achievement motivation inventory. 
The pattern indicates some overlap 
between instruments that presumably 
measure the same variable, but hardly 
inspire confidence in the unidimen- 
sionality of the achievement motiva- 
tion construct. The level of aspiration 
measures, besides being related to the 
criterion, are also related significantly 
to one or two other measures of 
achievement motivation. The TAT 
and manifest anxiety variables show 
little relationship to other measures. 
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TABLE 1 


INTERCORRELATIONS BETWEEN MEASURES OF ACHIEVEMENT MOTIVATION, 
ACE, anp GPA 


Achieve- 
ment 
Motiva- 
tion 
Inven- 
tory 


TAT o 
Achieve- 
ment 


Sentence 
Com- 


pletion 


Manifest 
Anxiety 


— .07 
— .07 


.40** | .13 


Adjective Checklist 
.35** 


Achievement Mo- 
tivation Inven- 
tory 

Sentence Comple- 
tion 

TAT n Achievement 

Aspiration-Grade* 
(first test) 

Aspiration-Grade* 
(last test) 

Manifest Anxiety | 


—.04 


ACE 


.27**|— 11 
.44** 


® Direction of scoring reversed by subtracting each discrepancy from a constant; the higher scores represented the 


* Significant at .05 level. 
** Significant at .01 level. 


The whole pattern of interrelation- 
ships was one that suggested not only 
the multidimensionality of the puta- 
tive achievement motivation con- 
struct, but also the probability that 
various measures of that construct 
might reflect quite different aspects of 
it and would therefore either be little 
correlated or selectively correlated 
with other measures and with the cri- 
terion. A factor analysis of all of the 
items of these various measures 
seemed admirably suited to shed light 
on such questions as these, and such 
an analysis was undertaken as the 
second step in the investigation.’ 


*The correlation matrix for the 29 indi- 
vidual achievement motivation items that 
were subjected to factor analysis has been 
deposited with the American Documenta- 
tion Institute. Order Document No. 6739 
from ADI Auxiliary Publications Project, 
Photoduplication Service, Library of Con- 
gress; Washington 25, D. C., remitting in 
advance $1.25 for microfilm or $1.25 for 
photocopies. Make checks payable to: Chief, 
Photoduplication Service, Library of Con- 
gress. 


Table 2 shows the rotated factor 
matrix which resulted when the indi- 
vidual items of the achievement mo- 
tivation measures were intercorrelated 
and factor analyzed. The items of the 
multiple-choice inventory, not in- 
cluded in the earlier analysis, were in- 
cluded in the present one to broaden 
the basis for analysis and comparison. 
Factorization was terminated when 
the factors isolated accounted for ap- 
proximately 100% of the estimated 
common factor variance. A factor was 
not interpreted, however, unless the 
product of its two highest loadings ex- 
ceeded two standard errors of a zero- 
order correlation for an N of the mag- 
nitude used in this study (Fruchter, 
1954, p. 79). Two criteria were used 
for the rotational procedure: the vari- 
max criterion (Kaiser, 1958), and the 
requirement that the final rotational 
solution achieve optimum factorial 
simplicity for the GPA—ACE crite- 
rion. After the first criterion was met, 
however, there was only one small ad- 


x 
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| 
on- 
M Grade* | Grade* | GPA- 
| test) test) ae 
15 | .02 | —.15 |—.02 | .36**| .40** 4 
.07 |—.20* .08 |—.23**|— .06 .18* 
.06 .09 |—.32**|—.17* .09 
13 | .18* .11 .08 |—.07 
-26**| | .20*| .48**) 
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.09 
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TABLE 2 
Rotatep Factor Matrix ror 29 Inpices or ACHIEVEMENT MoTIVATION 


Measures Il Ill IV V VI 


1. GPA-ACE 01 00 —20 08 
2. Aspiration-Grade Discrepancy* (first —07 | —32 —05 10 
test) 
3. Aspiration-Grade Discrepancy* (last —35 
test) 


Adjective Checklist Items: 
. Ambitious 33 
. Industrious 42 
. Hardworking 58 
. Studious 36 
. Competitive 08 
. Motivated 28 


Achievement Motivation Inventory Items: 

10. I am more ambitious than the average 
person. 

11. When I’m faced with difficult tasks, I 
generally “‘give up’’ too easily. 

12. I exert considerable effort to get good 
grades. 

13. I wish I were more successful as a stu- 
dent. 

14. I try very hard to do well in anything 
I undertake. 

15. I find that the standards I set for my 
own work are usually too high for me 
to attain. 

16. My parents exert considerable pres- 
sure on me to achieve good grades. 

17. I have strong motivation to succeed in 
my work. 

18. I feel under constant pressure to do 
well in my school work. 


19. Manifest Anxiety 
20. Sentence Completion 


Multiple-Choice Questionnaire Items: 

21. Do you feel that thus far you have 
really lived up to your parents’ ex- 
pectations for you? 

. Are your parents satisfied with your 
grades in college? 

. How hard do you work on your school- 
work in comparison with the typical 
student of your sex? 

. How important are grades to you? 

. Do you feel that your parents are sat- 
isfied with you as a person? 

. Are you satisfied with yourself as a 
person? 

. Do you and your parents agree on 
“‘values’’—what constitutes the 
really important things in life? 

. How would you rate your study habits 
in comparison with the average stu- 
dent of your sex? 


29. TAT n Achievement —01 | —02 | —09 05 00 | —06 | —4l1 


® Direction of scoring reversed by subtracting each discrepancy from a constant; the higher scores represented 


= | vil 
09 
=14 
| —05 
| 36| 20] 26 
43|-32| 17| 09 
| 2 | -19| -09| 18 

40| 04] O4| 16 

| 45| 08 
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4 -18 | -33| 23|-20| 23| 06 | —29 

01] 10] 49| —08 
49! 03| 40| 38|-17 
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13 | -36 | -09| -11| 09 

09| -03| 33/-26/ 14] 34| 07 
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|-03| 1] o1] 2% 
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ditional rotation that had to be made 
to meet the requirements of the sec- 
ond. The following factors were ten- 
tatively identified. 

Factor I had its highest loadings for 
the GPA—ACE criterion and for two 
variables (23 and 28) which reflect the 
actual effort expended to do an effec- 
tive academic job. Other variables 
have loadings to the extent that they 
also reflect an earnest application of 
effort to obtain tangible academic re- 
sults. This factor might best be called 
Academic Motivation and Efficiency. 
It is interesting to note that the varia- 
bles having the highest loadings on 
this criterion factor were objective 
self-rating indices, that the level of 
aspiration measures had moderate 
loadings on the factor, that the two 
projective measures (20 and 29) did 
not have loadings on this factor, and 
that neither manifest anxiety (19) nor 
self-satisfaction (26) had loadings on 
this important criterion factor. 

Factor II had its two highest load- 
ings for items relating to self-satisfac- 
tion and acceptance (25 and 26). It 
has negative loadings for manifest 
anxiety (19) and for the item express- 
ing a desire to be more successful as a 
student (13). It seems appropriate to 
refer to this factor as Self-Satisfaction. 
It is interesting to note the lack of re- 
lationship between this Self-Satisfac- 
tion factor and the GPA—ACE cri- 
terion. 

Factor III can best be interpreted 
in terms of its contrast with Factor I. 
It has no loading for the GPA—ACE 
criterion. It has negative loadings for 
the level of aspiration measures, in- 
dicative of its correlation with larger 
discrepancies between aspiration and 
achievement, as contrasted with the 
positive loadings of the first factor. It 
has a positive loading for the item ex- 
pressing the desire for more success as 
a student, in contrast to the first fac- 
tor’s negative loading for the same 
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item (13). Its loading of .04 contrasts 
with the first factor loading of .59 for 
the item expressing parental satisfac- 
tion with the subject’s grades (22). Its 
much smaller loadings for Items 23 
and 28, earlier interpreted as reflecting 
actual willingness to expend effort to 
achieve results, must be contrasted 
with the corresponding very high load- 
ings for the first factor. For most of the 
remaining variables there seems to be 
a similar tendency for low loadings to 
be associated with items referring ex- 
plicitly to effective application to aca- 
demic work, while the items with high 
loadings are more vague and general- 
ized, less behavior oriented, and more 
susceptible to a positive response set. 
All the evidence, and particularly that 
already noted for Items 1, 2, 3, and 13, 
suggests that Factor III should be in- 
terpreted as Wish-Fulfillment Moti- 
vation. The factor seems to represent 
strong desire but little application or 
fulfillment. 

It is interesting to compare the third 
factor loadings of the sentence com- 
pletion test with its first factor load- 
ings. The resulting contrast strongly 
suggests that projective test users must 
avoid jumping to conclusions about 
what level of experience or motivation 
is being tapped by a projective test. 
The appreciable loading of the sen- 
tence completion test on the Wish-Ful- 
fillment factor, its lack of loading on 
the first factor, and the earlier re- 
ported correlation of —.32 between 
sentence completion and ACE, all sug- 
gest that this projective may tend to 
identify the less able student who 
yearns for high achievement but who, 
because of discouragement or inertia, 
cannot really buckle down to achieve 
results. This hypothesis also receives 
partial confirmation from the original 
correlations. 

Factor IV has a high positive load- 
ing for the adjective “motivated” (9) 
and a high negative loading for the in- 
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ventory item indicating the subject's 
tendency to “give up” with difficult 
tasks (11). Several lesser loadings rep- 
resent similarly achievement oriented 
variables. What most distinguishes 
this factor, however, is its lack of load- 
ings for all items related strictly to 
academic motivation alone. Items hav- 
ing no loadings on this factor include 
two items concerned with willingness 
to work hard on schoolwork (12 and 
23), one concerned with the impor- 
tance of grades to the subject (24), 
and one concerned with the effective- 
ness of the subject’s study habits (28). 
Because of this specific exclusion of 
academic motivation from what seems 
to be a generally positive achievement 
orientation, it seems logical to desig- 
nate this factor as Nonacademic 
Achievement Motivation. Appropri- 
ately enough, this factor has no load- 
ing for the GPA—ACE criterion. 

Factor V is notable for its high load- 
ings for items concerned with feeling 
under parental pressure to do good 
academic work (16 and 18), aiming at 
standards that are too high to attain 
(15), but still not living up to parental 
expectations or achieving grades satis- 
factory to them (21 and 22). This fac- 
tor seems most clearly to define an Ex- 
ternal Pressure to Achieve. 

Factor VI, the most difficult factor 
to interpret, has a high loading for an 
item referring to the subject’s willing- 
ness to try hard in all he undertakes 
(14) and lesser but still appreciable 
loadings for items reflecting a gener- 
alized desire to achieve (10, 12, and 
17). Yet it is quite unrelated to the 
GPA-—ACE criterion. Unlike the third 
factor, it has no loading for the level 
of aspiration measures (2 and 3) or 
for the item reflecting a desire for more 
success as a student (13). Tentatively 
we will call this factor Imputed Gen- 
eralized Motivation without Attend- 
ant Effort. It is interesting to note that 
the sentence completion test again 


loads on a factor that involves desire 
to achieve but little evidence of real 
or appropriately directed effort. 

Factor VII should probably be in- 
terpreted as a residual or error factor 
of no importance. The product of its 
two highest loadings did not exceed 
the two standard error criterion. Its 
highest loading was for the TAT n 
Achievement test, whose appearance 
on this error factor is most probably 
due to low reliability. This same test 
had but one significant correlation in 
the entire battery, and this was low 
and perhaps attributable to chance. 

An analysis of this type would seem 
to have several implications requiring 
the thoughtful consideration of re- 
searchers in this area. First, it is evi- 
dent that the achievement motivation 
construct is not a unitary construct 
with invariable meaning, and that any 
attempt to regard it in this light is 
likely to lead to methodological con- 
fusion, conflicting results, and concep- 
tual stagnation. In this study it was 
possible to isolate five or six relatively 
independent dimensions which repre- 
sented different and psychologically 
significant aspects of the desire to 
achieve. It would therefore seem im- 
perative for each research worker in 
this area to offer some empirical evi- 
dence of which of several possible va- 
rieties of achievement motivation he 
had chosen to employ in his investiga- 
tion. To ignore the complexity of the 
achievement motivation construct is 
to render many results uninterpreta- 
ble. 

Secondly, it is apparent that tests 
and test items showing equivalent ini- 
tial promise may turn out to differ 
widely in their ability to measure any 
given aspect of achievement motiva- 
tion. Empirical surprises seem almost 
the rule rather than the exception. The 
TAT n Achievement test had the low- 
est communality of all 29 variables 
employed in the factor analysis, and 
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it loaded only on an error factor. Its 
one significant correlation in the origi- 
nal correlation matrix was a negative 
correlation with an item relating to 
ambition, and its correlation with the 
GPA-—ACE criterion was —.07. On the 
other hand, two individual objective 
items (23 and 28) had very high load- 
ings on the first factor and were cor- 
related .53 and .52, respectively, with 
the GPA—ACE criterion. This is a 
notable result, not only because such 
correlations are even difficult to 
achieve with whole tests, but also be- 
cause they were achieved with objec- 
tive, multiple-choice, self-rating items 
that focus on concrete, easily observed 
behaviors that are a part of everyday 
experience. 

Thirdly, tests and test items may 
differ widely in terms of which particu- 
lar aspects of the desire to achieve 
complex they actually measure. For 
example, the sentence completion test 
had a .09 loading on the first factor, 
a correlation of .09 with the GPA— 
ACE criterion, and significant corre- 
lations of —.32 and —.17 for the ACE 
and GPA, respectively. There is cer- 
tainly no evidence of its validity as a 
measure of achievement motivation 
here, and if GPA had been used as a 
criterion, results would be directly 
contrary to hypothesis. But the com- 
plete analysis revealed that this test 
was loaded on the Wish-Fulfillment 
factor, and its loading there gave cre- 
dence to Lazarus’ (Lazarus et al., 
1957) contention that there is no one- 
to-one relationship between the ex- 
pression of a need in fantasy and its 
actualization in behavior. Indeed, the 
fantasy may even serve as a substitute 
for overt striving. Analyses of fantasy, 
then, may not always be “depth” anal- 
yses; they may actually reveal rela- 
tively superficial aspects of behavior 
and personality. 

Certain objective items also turned 
out to measure quite different aspects 
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of the desire to achieve complex than 
had been originally anticipated. It had 
been anticipated, for instance, that 
Items 8, 11, and 18 would be highly 
related to the criterion. As it devel- 
oped, they were not at all related to 
the criterion nor to the first factor; 
they were, however, loaded on other 
factors. Items 22 and 28, on the other 
hand, were highly related to the 
GPA-—ACE criterion, having correla- 
tions of .50 and .52, respectively, but 
their patterns of loadings in the factor 
matrix were quite different. Here, too, 
cautious analysis is needed to ascer- 
tain just what aspect of the desire to 
achieve complex is actually being 
measured. 

The importance of such analysis is 
particularly apparent in comparing 
the lack of correlation between some 
of the instruments represented in 
Table 1 with the relatively high load- 
ings of the factor matrix and the rela- 
tively high interitem correlations of 
the original correlation matrix. A com- 
parison of this kind furnishes compel- 
ling evidence that intertest correla- 
tions can be attenuated by items 
representing different factorial dimen- 
sions or by same-test items actually 
counteractive in their influence. 

Fourthly, it is not only important 
to acknowledge that different tests and 
test items may represent different as- 
pects of the desire to achieve complex; 
it is equally important to recognize 
that the same test or test item may 
represent several different aspects of 
the desire to achieve complex. The ad- 
jective “industrious” (9), for instance, 
has loadings on four different factors 
and could represent four distinctly 
different attitudes toward achieve- 
ment. There are several other items 
(e.g., 9, 17, and 23) that are loaded on 
two or three factors. It is apparent 
that identical responses can have very 
different meanings for different sub- 
jects. 
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Finally, there were certain by-prod- 
ucts of this investigation that are 
worthy of note despite their peripheral 
status. The pattern of loadings for 
manifest anxiety did not lend confir- 
mation to the hypothesized drive at- 
tributes of that variable, but it must 
be acknowledged that this investiga- 
tion was not designed as a critical test 
of that issue. It is also interesting to 
note that the item relating to self- 
satisfaction (25) was correlated .00 
with the GPA—ACE criterion and .09 
with GPA alone. The Self-Satisfaction 
factor had similarly insignificant and 
negligible loadings for items concerned 
with strictly scholastic motivation (7, 
17, 23, and 24). Apparently the self- 
concepts of our college students are 
not nearly as influenced by matters re- 
lating to academic motivation and 
achievement as we are sometimes in- 
clined to believe. 


SUMMARY 


The purpose of this study was to 
determine whether “achievement mo- 
tivation” was a unitary construct with 
invariable meaning or a complex of 
relatively independent dimensions 
with different implications and differ- 
ent empirical correlates. Twenty-nine 
indices of achievement motivation 
were subjected to factor analysis. Six 
factors were identified: Academic Mo- 
tivation and Efficiency (the only fac- 
tor highly predictive of actual aca- 
demic performance) , Wish-Fulfillment 
Motivation, Nonacademic Achieve- 
ment Motivation, Self-Satisfaction, 
External Pressure to Achieve, and Im- 
puted Generalized Motivation without 
Attendant Effort. In addition to dem- 
onstrating that there are several va- 
rieties of achievement motivation, the 
results also indicated that: tests and 
test items differed widely in their abil- 
ity to measure any of the several facets 
of this complex variable, identical re- 
sponses could have very different 


meanings for different subjects, self- 
rating procedures excelled projective 
methods in predicting actual academic 
performance, and the projectives were 
either loaded on wish-fulfillment or er- 
ror factors. 
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Studies dealing with the perceptual 
and conceptual capacities of deaf chil- 
dren have suggested that children deaf 
from birth fail to develop the ability 
to deal with abstract material in a 
fashion comparable to hearing chil- 
dren (Larr, 1955; Myklebust, 1953; 
Oleron, 1950; Templin, 1950). Other 
studies, however, have shown tht the 
differences between the thinking proc- 
esses of deaf and hearing children do 
not correspond to the differences be- 
tween the concreteness of the aphasics 
and the abstract functioning of normal 
subjects (Oleron, 1953) but correspond 
much more closely to the differences 
between a younger and older group of 
hearing children (Clarke School, 
1940). When the level of verbalization 
required is within the range of vocabu- 
lary and experience of the deaf sub- 
ject, deaf subjects perform as well as 
hearing subjects on abstract as well as 
less abstract tasks (Rosenstein, 1959). 

There are three major purposes of 
this experiment. First, we are inter- 
ested in investigating a specific aspect 
of the cognitive processes of deaf and 
hearing children, that of categoriza- 
tion. Categorization, “to render dis- 
criminably different things equiva- 
lent,” forms one of the bases for all 
other cognitive activities (Bruner, 
Goodnow, & Austin, 1956, p. 1). Our 
second major concern is in distinguish- 
ing between the process of categoriza- 
tion on the one hand, and the process 
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of verbalization on the other, and to 
judge the adequacy of categorization 
independently of the adequacy of ver- 
balization. The process of learning may 
involve both learning a new verbal 
attribute and then learning the non- 
verbal category that goes with it as 
well as learning a verbal attribute to 
go with a nonverbal category already 
formed (Brown, 1958). Our third con- 
cern is in attempting to separate the 
effects of deafness from the cognitive 
effects of age and achievement level. 

The general hypothesis of this study 
was: There will be no differences in 
categorization or verbalization found 
for one comparison of deaf subjects 
with hearing subjects of matched age 
and IQ that will also be found for the 
comparison of these deaf subjects with 
hearing subjects matched on school 
achievement and IQ. 


METHOD 


Subjects. One group consisted of five girls 
and three boys at the Clarke School for the 
Deaf in Northampton, Massachusetts, rep- 
resenting the total number of graduating 
seniors with very profound hearing loss. 
Their training was by the oral method of 
instruction. Their average IQ on the WAIS 
(both verbal and performance subscales) 
was 10263; SD, 693. On the Stanford 
Achievement Test, their average achieve- 
ment was equivalent to the 7.79 grade level. 
Their mean age was 18.09 years; SD, 0.33. 

Our second group was composed of eight 
junior high school students, five girls and 
three boys. Their average Stanford Achieve- 
ment level was equivalent to the 7.68 grade 
level. Their average IQ, based on WISC 
and Terman-McNemar, was 10463; SD, 
5.05. Their average age was 142 years. 

Our third group included eight high 
school seniors, five girls and three boys. 
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Their average age was 17.97 years; SD, 0.50. 
Their average IQ, based on WAIS and Ter- 
man-McNemar, was 106.38; SD, 5.50. These 
subjects were achieving at an average level 
for their high school class, and were expected 
to graduate without difficulty. The high 
school and junior high school students were 
screened carefully in consultation with their 
teachers and counselors, and no subject was 
included with any atypical personality and 
physical characteristics. 

There were no significant differences be- 
tween the deaf and the junior high school 
subjects on achievement level and IQ, and, 
similarly there were no significant differ- 
ences between the deaf and the high school 
students on age and IQ. The size of the 
sample makes it necessary for this study to 
be regarded as exploratory and, of course, 
subject to replication. 

Categorization and Verbalization Tests. 
The Goldstein-Scheerer (1941) Object Sort- 
ing Test was individually administered. This 
test consists of 33 everyday objects includ- 
ing such things as a real cigar, a real pair of 
pliers, a toy hammer, etc. (Rapaport, Gill, 
& Schafer, 1945). 

Procedure. The Object Sorting Test was 
administered in three parts: the free, active, 
and compliant sections. In the free sorting 
section, the subject was asked to put all the 
objects into any number of groups and then 
tell why the objects in each group belonged 
together. In the active section, the subject 
was asked to put all the objects together 
that belonged with a preselected sample ob- 
ject and give his reasons for the categoriza- 
tion. There were eight “target” objects, the 
first selected by the subject and rest pre- 
sented by the experimenter. The compliant 
section involved the presentation of 12 dif- 
ferent preselected groupings of objects and 
asking each subject to give the reasons for 
the groupings. All three sections of the test 
were administered in one session and all 
subjects received the free, active, and com- 
pliant sections in order. Written instruc- 
tions were used for the deaf subjects and 
the same instructions were given orally to 
the hearing subjects. There was no doubt 
on the part of the experimenters that all 
verbalizations of all subjects were well un- 
derstood. 

Scoring. A scoring system for categoriza- 
tions was devised, based partially on that 
used by Rapaport et al. (1945, pp. 401-406). 
Classification of verbalizations were based 
upon those used by Rapaport as well as 
upon the criteria for types of rules of ver- 
balization set up by Bruner et al. (1956, pp. 
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4-7). Two independent scorers agreed on 
95% of a sample of 350 items scored inde- 
pendently. Copies of the complete scoring 
system are available upon request. 

Statistical Procedure. Our statistical anal- 
ysis compared separately the deaf subjects 
with each one of the two control groups; the 
question of the overall differences among 
the three groups was not a concern of this 
study. 

To make some data comparable from 
subject to subject, proportions were calcu- 
lated for measures such as adequacy of 
categorization and verbalization. Estimate 
tests of homogeneity of variance were per- 
formed and heterogeneous data were con- 
verted to 7 scores (Edwards, 1954). All 
variances were then found to be homo- 
geneous. 


RESULTS AND Discussion 


Prediction 1. The deaf subjects will 
have a similar number of divisions and 
categories as the hearing subjects 
matched on achievement and IQ but 
significantly more categories and di- 
visions than the hearing subjects 
matched on age and IQ. Furthermore, 
the deaf subjects will not differ in the 
number of failures from either hear- 
ing group. Divisions are defined as the 
number of groupings and/or single ob- 
jects that the 33 objects are divided 
into in the free sorting section. The 
number of categories is equal to the 
number of groupings with two or more 
nonidentical objects included. A fail- 
ure consists of an object that is not 
grouped with another object or of an 
absolute identity class such as two 
sugar cubes. 

Results. The results, in general, sup- 
port this prediction. Deaf subjects 
have the same number of divisions, 
categories, and failures as both groups 
of hearing subjects (Table 1), and 
thus appear as effective as hearing 
subjects in the two control groups in 
reducing the complexity of this ex- 
perimental world. 

Prediction 2. The deaf subjects will 
categorize as adequately but will not 
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TABLE 1 
Test Resvutts ror COMPARISONS BE- 
TWEEN Dear AND CONTROL GROUPS 
MatcHep ON AGE AND IQ (C;) AND ON 
STANFORD ACHIEVEMENT AND IQ (C2) 


1 0.031 
izations used |Deaf |52.18 | 6. | 
7 | 5. 


47.8 1.340 


inadequately? 


Comparison Group M SD 
Divisions C, 8.75 | 1.79 }0.298 
Deaf | 9.00 | 1.32 
C: 7.88 | 1.60 |1.460 
Categories C, 7.13 | 1.05 0.182 
Deaf | 7.25 | 1.39 
C: 6.50 | 1.22 {1.070 
Failures C, 2.13 | 1.97 |0.00 
Deaf | 2.13 | 0.93 
C2 2.25 | 1.92 (0.155 
Concrete ver- 0.080) 0.078)1.418 
balization® Deaf | 0.158) 0.123 
C2 0.212 
Functional ver- |C,; 0.214) 0.222:0.404 
balization> |Deaf | 0.176) 0.113) 
C2 0.275) 0.123/1.571 
Category width |C, 5.00 | 1.16 |2.239* 
with failures |Deaf | 3.65 | 1.10 
C2 4.13 | 0.72 \0.968 
Category width |C, 5.20 | 1.03 |2.482* 
without fail- |Deaf | 3.79 | 1.09 
ures 4.51 pa 
Formal verbal- |C, [52.33 11.07 
15 
| 88 


 /47.35 | 9.18 (0.898 
[Deaf {52.33 [11.45 | 
{50.37 | 7.42 (0.380 


Inclusive ver- 
balization? 


* The ¢ value is the result of the comparison made of 
the deaf group mean with the control group mean just 
opposite the corresponding ¢ value. 

> Raw scores converted to proportion of total number 
of verbalizations. 

° Proportion of formal inadequate to total inadequate 
converted to T scores. 

4 Proportion of inclusive verbalization to total num- 
ber of verbalizations converted to T scores. 

* Significant at p < .05 level. 


have as many adequate verbalizations 
as the hearing subjects matched on age 
and IQ. Furthermore, the deaf sub- 
jects will have the same number of 
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adequate verbalizations as the group 
matched on achievement and IQ. An 
adequate categorization is one in 
which all the objects included are rele- 
vant to each other independent of the 
verbalization, and there are no rele- 
vant objects present in the sample that 
are not included in the grouping. An 
adequate verbalization is defined as 
one that covers correctly the group 
sorted. An inadequate verbalization 
may be false, or may not take into ac- 
count all the objects included in the 
categorization, or may take into ac- 
count all objects included in the cate- 
gorization but pertain to objects out- 
side the categorization for which the 
verbalization is given. 

Results. The results support this 
prediction. Analyses of variance per- 
formed on the proportion of adequate 
categorizations to the opportunities for 
categorization show that there are no 
differences between the deaf subjects 
and either of the two hearing groups 
(Tables 2 and 3). When the task of 
grouping equivalent objects together 
is judged indeper:dently of the verbali- 
zation of the reason for that grouping, 
deaf subjects perform as well as hear- 
ing subjects of their own age and IQ. 
Furthermore, the deaf subjects have 
significantly fewer adequate verbali- 
zations than the hearing group 
matched on age and IQ (F = 6.58, 
p < .05) but there are no differences 
between the deaf group and the hear- 
ing group matched on achievement and 
IQ (Tables 2 and 3). Thus, in terms 
of adequacy of verbalization, the deaf 
subjects seem to resemble the younger 
hearing subjects. 

Prediction 3. The deaf subjects will 
have a greater number of adequate 
categorizations accompanied by inade- 
quate verbalizations than the control 
group matched on age and IQ while 
not differing from the hearing group 
matched on achievement and IQ. 
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TABLE 2 
MEANS AND STANDARD DEVIATIONS OF THE 
Dear GRouP AND THE Two ConTROL 
Groups 


a 
S 


Variable 
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TABLE 3 


ANALYsis OF VARIANCE ResvuLts For Com- 
PARISONS BETWEEN DEAF AND THE CoNn- 
TROL Group MatTcHED ON AGE AND IQ 
(C,) AND on STANFORD ACHIEVEMENT 
and IQ (C2) 


Adequacy of cate- 
gorization 


Adequacy of ver- 
balization 


Adequate categor- 
ization—inade- 
quate verbaliza- 
tion 


SSS 


Formal _ generic 
verbalization 


Total formal ver- 
balization 52.53 


45.82 


63° 
52.05 
50.65 


Proneness to 
change 


NOD 


® Proportion of the number of adequate categories to 
total opportunities for categorization. 

> C, is the control group matched with the deaf on 
age and IQ. 

© Cy is the control group matched with the deaf on 
Stanford Achievement and IQ. 

4 Proportion of the number of adequate verbaliza- 
tions to the number of categorizations. 

* Converted to T scores. 


Results. The analyses were con- 
ducted on standard scores converted 
to T scores (Tables 2 and 3). The re- 
sults indicate that the deaf subjects 
have a significantly greater number 
of adequate categorizations accom- 
panied by inadequate verbalizations 
than the control subjects matched on 
age and IQ (F = 5.47; p < .05). But 
there were no differences between the 
deaf subjects and those matched with 
the deaf on achievement and IQ. The 
results support our prediction. It may 
be pointed out that the younger con- 
trol group had significantly more ade- 


5.474° 


0.009 


11.59 | 11.59 
2622.46 | 187.32 
385.90 | 385.90 
1755.12 | 125.37 


93.61 | 93.61 
14 |1670.42 | 119.32 
1 32.46 32.46 


4 85.05 


* Raw scores converted to proportion of adequate 
categorizations to total number of categorizations. 

> Raw scores converted to proportion of adequate 
verbalizations to total number of verbalizati ( ber 
of divisions plus 8 active plus 12 compliant). 

© Raw scores converted to T scores. 

4 Proportions of formal generic verbalizations to to- 
tal number of verbalizations converted to T scores. 

® Propertions of total formal verbalizations to total 

ber of verbalizati converted to T scores. 
* Significant at p < .05 level. 


3 | 
fh 
Group M = 
| r Deaf | 0.713* Variable a SS MS P 
tion* 
Deaf | 0.6244 C; Between treatment | 1 | 0.014 0.014/0.280 
; C, 0.774 groups 
Cs 0.582 Error 14 0. 0.050 
Cs Between treatment | 1 | 0.013) 0.013/0.317 : 
if Deaf |52.48* 
C, 45.32 Error 4 0.573) 0.041 
C: 52.83 Adequacy of verbaliza- 
tion” 
C; Between 1 0.270; 0.270/6.585° 
Deaf |52.70¢ Error 14 | 0.573) 0.041 
Cc 51.99 C1 Between 1 0.021} 0.021/0.438 
ization® 
C2 C; Between 1 | 410.06 | 410.06 
Error 14 |1048.76 | 74.91 
Deaf C1 Between 1 | 0.96| 0.96 
C; Error 14 |1544.34 | 110.31 
| Formal generic verbal- 
Between 1 5.98 | 5.98 |0.003 
Error 14 [3064.02 | 218.86 
3 C: Between 1 | 645.55 | 645.55 |3.417 
Error 14 2644.15 | 188.82 
Total formal verbalisa- 
tion® 
C, Between 1 | 0.619 
Error 14 
Between 1 3.078 
Error | 
| : 
Proneness to Change® | 
C, Between 0.785 
Error 
aa C: Between 0.382 
uk 
Ths 
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quate sortings accompanied by inade- 
quate verbalizations than the older 
control group. Our results suggest that 
deafness holds back the process of cor- 
rectly relating verbal attributes to 
nonverbal categories but that this re- 
tardation is similar to that shown by 
younger hearing subjects when they 
are compared to older hearing subjects. 

Prediction 4. The deaf subjects will 
have more functional, more total con- 
crete, fewer formal, fewer formal ge- 
neric verbalizations than the control 
group matched on age and IQ but the 
same number as the control group 
matched on achievement and IQ. A 
functional verbalization specifies a 
rule of categorization based upon the 
sharing of a common function. Total 
concrete verbalizations include the 
syncretistic, fabulatory, chain, sym- 
bolic, split narrow verbalization as de- 
fined by Rapaport et al. (1945), and 
formal specific verbalization type. 
Formal verbalizations are defined as 
verbalizations specifying a rule of 
grouping based on the sharing of an 
intrinsic property (“all red,” or “all 
kitchen utensils”) among the objects 
in the group (Bruner et al., 1956). A 
formal generic verbalization specifies 
a rule of grouping based upon the shar- 
ing of highly complex and class-like 
attributes (“all metal’’). 

Results. There are no differences be- 
tween the deaf and either of the hear- 
ing control groups with respect to any 
of the verbalization types discussed 
above (Tables 1, 2, and 3). This pre- 
diction is not supported. However, the 
group matched with the deaf on 
achievement and IQ had significantly 
more “total concrete” verbalizations 
than the older control group. This re- 
sult indicates that the test does differ- 
entiate, at least in regard to concrete 
verbalizations, levels of verbalization 
due to age and achievement differences 
in hearing subjects. On the whole these 
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concrete verbalizations represent an 
inability to utilize a conjunctive rule 
of grouping to explain a categoriza- 
tion. Further, they can also be de- 
scribed as “private” verbalizations 
(McGaughran & Moran, 1956). Deaf 
subjects on these tasks are no more 
concrete verbally, do not demonstrate 
any greater inability to utilize con- 
junctive rules of grouping, and utilize 
no more private verbalizations than 
the two control groups. 

There is little evidence and theory 
concerning the following issues, and 
they are stated in the form of a ques- 
tion. 

Are there any differences between 
the deaf subjects and either control 
group in proneness to change, in cate- 
gory width, in number of formal ver- 
balizations used inadequately, and in 
the number of too inclusive verbaliza- 
tions? Proneness to change is meas- 
ured by the exclusion of objects in the 
active sorting which were included 
with these target objects in free sort- 
ing or by the inclusion, with the target 
objects in the active sorting, of objects 
not included with them in the free 
sorting. Category width is measured 
by the number of objects included with 
the target objects in the active section, 
both counting and not counting fail- 
ures. 

The data were converted to T' scores 
and analysis showed that there are no 
differences between deaf subjects and 
either of the two control groups in 
overall proneness to spontaneous 
change (Tables 2 and 3). These results 
are in contrast to other results which 
have suggested that deaf subjects are 
more rigid than hearing children (Mc- 
Andrew, 1948). The question of the 
rigidity of the deaf is nota settled one, 
since our evidence points to no differ- 
ence between groups when spontaneous 
behavior is examined. 

Our results show that the deaf sub- 
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CATEGORIZATION AND VERBALIZATION IN DEAF ADOLESCENTS 


jects have narrower categories than 
the control group matched on age and 
IQ but do not differ in category width 
from the hearing subjects matched on 
achievement and IQ (Table 1). The 
use of the narrower categories by the 
deaf subjects who, at the same time 
have the same number of categories as 
the hearing groups, may indicate a 
greater difficulty in organizing the 
world and in generalizing rules of 
grouping to a wide array of objects. 
This, of course, would also be true of 
the younger group of hearing subjects 
from whom the deaf did not differ in 
category width. However, with respect 
to this small array of familiar objects, 
the use of narrower categories might 
be more appropriate and more predic- 
tive than the use of wider categories. 
The problem of the relation of deaf- 
ness to category width seems to be one 
that deserves future attention. 

The two proportions of formal in- 
adequate verbalizations and of inclu- 
sive verbalizations to the total number 


of verbalizations, respectively, were 
converted to 7 scores and were com- 
pared for the three groups. The deaf 
did not differ significantly from either 
control group in formal inadequate 
verbalizations and in inclusive ver- 
balizations, respectively (Table 1). 


SUMMARY 


This study has investigated the cat- 
egorization and verbalization proc- 
esses of 24 adolescents, 8 of whom were 
profoundly deaf. The remaining sub- 
jects comprised two hearing groups, 
one group matched with the deaf sub- 
jects on age and IQ and the other 
group matched on Stanford Achieve- 
ment and IQ. The Goldstein-Scheerer 
Object Sorting Test was administered 
and the deaf subjects were separe’ ely 
compared with each of the two cor trol 


groups. 
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Our conclusions are as follows: 

1. Deaf subjects categorize, on this 
task, as adequately as the hearing 
subjects. They have the same number 
of divisions, failures, and categories 
as the hearing subjects. On categoriza- 
tion tasks judged independently of 
verbalization, deaf subjects perform as 
well as hearing adolescents. 

2. Deaf subjects have more inade- 
quate verbalizations than hearing sub- 
jects and more adequate categoriza- 
tions accompanied by inadequate 
verbalizations than hearing subjects 
matched on age and IQ. These differ- 
ences are not true for the comparison 
of the deaf subjects and the hearing 
subjects matched on achievement and 
1Q. It is proposed that deafness retards 
learning of the correct relation be- 
tween the proper verbal attribute and 
the nonverbal category but that this 
retardation lies along a normal devel- 
opmental progression. 

3. Deaf subjects do not differ from 
hearing subjects in the type of verbali- 
zation or in the developmental level of 
the verbalizations used, or in their 
spontaneous changes of categorization 
throughout the test. 

4. Deaf subjects have narrower cate- 
gories than hearing adolescents of 
their own age and IQ though they do 
not differ from hearing subjects of the 
same achievement and IQ. 

5. There were no differences found 
between deaf and hearing subjects that 
indicated a definite shift in categori- 
zation ability distinct from the effects 
of age and achievement. 


REFERENCES 


Brown, R. Words and things. Glencoe, IIl.: 
Free Press, 1958. 

Bruner, J. 8., Goopnow, J. J., & Austin, G. 
A. A study of thinking. New York: 
Wiley, 1956. 

Cuarke ScHoo. ror tHe Dear, Clarence W. 
Barron Research Department, Psycho- 
logical Division. Studies in the psychol- 


’ 
a 
2 | 
or 
4 
7 


KATES, KATES, MICHAEL, AND WALSH 


ogy of the deaf. Psychol. Monogr., 1940, 
52(1, Whole No. 232). 

Epwarps, A. L., Statistical methods for the 
behavioral sciences. New York: Rine- 
hart, 1954. 

K., & Scueerer, M., Abstract and 
concrete behavior: An _ experimental 
study with special tests. Psychol. 
Monogr., 1941, 53(2, Whole No. 239). 

Larr, A. L. An experimental investigation 
of the perceptual and conceptual abili- 
ties of children in residential schools 
for the deaf. Unpublished doctoral dis- 
sertation, Syracuse University, 1955. 

McAnprew, H. Rigidity and isolation: A 
study of the deaf and the blind. J. 
abnorm. soc. Psychol., 1948, 43, 476- 
494. 

McGavueuran, L. S., & Moran, L. J. “Con- 
ceptual level” vs. “conceptual area” 
analysis of object-sorting behavior of 
schizophrenic and nonpsychiatric groups. 


J. abnorm. soc. Psychol., 1956, 52, 43- 

51. 

tesvust, R., Towards a new understand- 
ing of the deaf child. Amer. Ann. Deaf, 
1953, 98, 345-357. 

Ouzron, P. A study of the intelligence of the 
deaf. Amer. Ann. Deaf, 1950, 95, 179- 
195. 

Ougeron, P. Conceptual thinking of the deaf. 
Amer. Ann. Deaf, 1953, 46, 299-315. 
Rapaport, D., Gri, M., & Scuarer, R. Diag- 
nostic psychological testing. Vol. 1. 
Chicago: Year Book Publishers, 1945. 

Rosenstsin, J. Perception and cognition in 
deaf children. Unpublished doctoral dis- 
sertation, Washington University, 1959. 

Temptin, M. The development of reasoning 
in children with normal and defective 
hearing. Minneapolis: Univer. Min- 
nesota Press, 1950. 


(Received September 6, 1960) 


4 
= 
met 
- 
if 
4 | 
Al 
4 
| 
din 


Journal of Educational Psychology 
1961, Vol. 52, No. 4, 195-198 


CROSS-VALIDATION GROUPS, EXTREME GROUPS, AND 
THE PREDICTION OF ACADEMIC ACHIEVEMENT 


LEONARD J. KAZMIER' 
University of Notre Dame 


In two parallel studies, scores on the 
15 personality variables of the Ed- 
wards Personal Preference Schedule 
(EPPS) (Edwards, 1954) have been 
compared for over- and underachiev- 
ing college students (Gebhart & Hoyt, 
1958; Krug, 1959). In both studies, ex- 
treme over- and underachievers were 
selected from a larger sample of male 
college students, so that the middle 
range of achievers was eliminated from 
the study. Accordingly, analyses of 
variance were performed to test for 
the significance of differences between 
the extreme groups, but validity co- 
efficients were not computed. Though 
the scales that were found to discrimi- 
nate between subjects who over- or 
underachieved in relation to measured 
aptitude were not identical, the results 
of the two studies did agree to a con- 
siderable extent. In the Gebhart and 
Hoyt study, overachievers scored 
higher than underachievers on the 
Achievement, Order, Intraception, and 
Consistency scales, and significantly 
lower on the Nurturance, Affiliation, 
and Change scales. In the study by 
Krug, overachievers scored signifi- 
cantly higher on the Achievement, Or- 
der, and Endurance scales and signifi- 
cantly lower on Affiliation and 
Heterosexuality. The .05 level was 
defined as significant in both studies. 

There has come to be considerable 
concern about the widespread use of 
extreme groups in hypothesis testing. 
As McNemar has indicated (1960), 
such tests of significance are mislead- 
ing and may represent only a small de- 
gree of relationship between the two 


*Research conducted while the author 
was at Ohio State University. 


variables. The primary purpose of the 
present study was to investigate the 
relationship between the EPPS scales 
and a criterion of academic achieve- 
ment when scores of all subjects are 
utilized, not just those of selected ex- 
treme groups. The correlation tech- 
nique was used throughout because it 
provides a measure of the degree of 
relationship as well as its significance. 
To ascertain that the measured rela- 
tionships of the several scales with the 
criterion are not due to chance factors, 
a cross-validation group was included 
in the study. 


PROCEDURE 


The sample consisted of 140 students en- 
rolled in two sections of introductory psy- 
chology at Ohio State University during the 
Spring Quarter of 1960. Both sections were 
taught by the same instructor, the author of 
the present report. For use as a predictor, 
six paragraphs of material, including a total 
of 36 questions on the contents thereof, 
were chosen from old forms of the Ohio 
State Psychological Examination (Buros, 
1959, pp. 494-495). The criterion for success 
in the psychology course was the total num- 
ber of points accumulated at the end of the 
quarter from three departmental examina- 
tions and four class quizzes. 

In order to achieve comparability of the 
experimental and cross-validation groups, 
all class cards for the subjects were segre- 
gated by sex, and then categorized accord- 
ing to college year. Within the college year 
the cards were arranged in order of per- 
centile scores on the Ohio State Psycho- 
logical Examination. Seven students for 
whom these scores were not available were 
arbitrarily placed at the end of their respec- 
tive college-year groups, after the highest 
percentile scores. Then the overall sample 
was subdivided into two groups of 70 sub- 
jects each by alternatively putting one card 
in the experimental group and the following 
card in the cross-validation group. Com- 
parability of the two groups in terms of 
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their standing on the reading comprehension 
test, the criterion, and the 15 EPPS scales 
was ascertained. 

The correlation of each of the 15 scales 
with the criterion of success was determined 
for each of the groups and for the entire 
sample, and the Wherry-Doolittle test se- 
lection technique (Garrett, 1958, pp. 426- 
440) was then applied to the experimental 
group of subjects in order to determine the 
best selection and weighting of EPPS scales 
for maximum predictiveness. The applica- 
bility of this result to the cross-validation 
group was investigated by computing a 
composite r for that group by using the 
scales and weights derived from the experi- 
mental group. 


RESULTS 


The split-half reliability coefficient 
(odd vs. even) for the reading com- 
prehension test is .813, while the coef- 
ficient of correlation between this pre- 
dictor and the criterion of total points 
in the psychology course is .593, which 
is significant at the .001 level for the 
140 subjects involved in this study. 


TABLE 1 


CoMPARISONS BETWEEN THE EXPERIMENTAL 
AND Cross-VALIDATION GROUPS 


Experi- 
mental Cross-Validation 
Group Group (N = 70) 
Measure (N = 70) 
M | SD\|M SD| F 
Points in psychology |250.00/39.25 |260.10|36.05 | —* 
course 
Predictor Score 17.18) 6.22 | 16.84) 5.98 | — 
EPPS Scales: 
Achievement 13.67) 3.94 | 14.23) 4.75 | — 
Deference 11.41) 3.71 | 12.21) 3.67 | 1.64 
Order 9.44) 4.17 | 10.67) 3.90 | 3.24 
Exhibition 14.34) 3.33 | 14.34, 3.65 | — 
Autonomy 12.86) 4.20 | 12.66) 5.04) — 
Affiliation 16.26) 4.38 | 16.57, 4.49 | — 
Intraception 17.26) 4.88 | 17.34, 4.78 | — 
Succorance 11.38) 5.25 | 11.00) 4.13 | — 
Dominance 15.58; 4.68 | 14.41) 5.23 | 1.95 
Abasement 14.31| 4.99 | 14.70] 5.67 | — 
Nurturance .. 15.66) 5.18 | 16.18) 4.81 | — 
Change 17.57) 4.78 | 16.90, 4.55 | — 
Endurance 13.63) 5.67 | 13.08) 6.27) — 
Heterosexuality 14.97) 5.08 | 15.08) 5.58 | — 
Aggressiveness 11.00) 4.62 10.44 454) — 
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TABLE 2 
CoRRELATION OF THE EPPS Scales WITH 
THE CRITERION OF Success 
In PsycHOLOGY 


Cross 
= 
= 10) | |W = 
Achievement .113 -191 .153* 
Deference .013 |—.109 |—.043 
Order .029 |—.214*|— .080 
Exhibition .027 .157 .092 
Autonomy .073 |—.040| .013 
Affiliation —.139 |—.161 |—.149* 
Intraception .022 .099 | .059 
Succorance — .205* .078 |— .087 
Dominance -209* | .226*| .213°* 
Abasement —.054 |—.068 |—.060 
Nurturance —.167 |—.163 |—.164* 
Change —.244**| |—.114 
Endurance .088 |—.061 .018 
Heterosexuality .192 -037 | .114 
Aggressiveness -115 |—.034 .043 
*p< .10. 
"p< 


That the procedure used to divide the 
subjects into experimental and cross- 
validation groups did result in com- 
parability of the groups in many im- 
portant respects is indicated by the 
data in Table 1, which presents the 
mean scores and standard deviations 
for the two groups on the reading com- 
prehension test, the criterion, and the 
15 EPPS scales. Not one of the ob- 
tained F ratios is significant at the .05 
level. 

The correlation coefficients between 
the criterion of success in the psychol- 
ogy course and each of the 15 scales of 
the EPPS are indicated in Table 2 for 
the experimental and cross-validation 
groups as well as for the total sample. 
For the experimental group, the corre- 
lation coefficients for Succorance and 
Dominance were significant at the .10 
level, while that for Change was sig- 
nificant at the .05 level. The cross- 
validation group yielded a somewhat 
different pattern of correlations, re- 
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PREDICTION OF ACADEMIC ACHIEVEMENT 


sulting in significance at the .10 level 
for the Order and Dominance scales. 
For the total sample of subjects, the 
correlation coefficients for Achieve- 
ment, Affiliation, and Nurturance were 
significant at the .10 level while that 
for Dominance was significant at the 
.05 level. 

It would appear that the desired sta- 
bility of relationship of the EPPS 
scales with the criterion is lacking in 
this study. For example, of the three 
scales whose relationship with the cri- 
terion was significant at the .10 level 
or better in the experimental group, 
only one scale “held up” for the cross- 
validation group, while the algebraic 
signs of the correlation coefficients for 
the other two groups actually were 
reversed. In order to verify this con- 
clusion statistically, the Wherry-Doo- 
little technique was utilized to ascer- 
tain the best selection of scales for the 
experimental group and the appropri- 
ate beta weights to be used in conjunc- 
tion with each scale. Stability of the 
scales is then indicated by the extent 
to which these beta weights apply for 
use with the cross-validation group. As 
indicated by Table 3, for the experi- 
mental group it was found that the use 
of the Change, Succorance, and Het- 
erosexuality scales results in the maxi- 
mal coefficient of multiple correlation 
corrected for shrinkage. Addition of 
the fourth best scale, Abasement, re- 
sults in a decrease in the magnitude of 
the correlation coefficient. 

The beta weights obtained by the 
Wherry-Doolittle test selection method 
are —.252 for Change, —.233 for 
Suececorance, and .181 for Hetero- 
sexuality. Utilizing these weights de- 
rived for the experimental group and 
the correlation coefficients of the 
chosen scales with the criterion in the 
cross-validation group results in a 
composite coefficient of correlation for 
the cross-validation group of —.058. 
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TABLE 3 
Best SeLectTion or EPPS ScaLes FOR THE 
EXPERIMENTAL GROUP BY THE WHERRY- 
Metuop 


Scales 


oR 
244 
.313 
345 

331 


Change 

Change + Succorance 

Change + Succorance + Hetero- 
sexuality 

Change + Succorance + Hetero- 
sexuality + Abasement 


Thus, the validity evidenced for the 
EPPS scales in the experimental group 
did not at all hold up in the cross-vali- 
dation group, in that the scales and 
weights that resulted in a multiple R 
of .345 for the experimental group 
yielded a slightly negative composite 
coefficient for the second group. 


SUMMARY 


Previous studies have reported pre- 
dictiveness for certain scales of the 
Edwards Personal Preference Schedule 
(EPPS) in regard to academic over- 
and underachievement when the anal- 
ysis of variance technique was applied 
to selected extreme groups. The pur- 
pose of the present study was to test 
out the EPPS in a situation in which 
no subjects are eliminated from the 
study, the correlation technique is 
used, and both experimental and cross- 
validation groups are included. 

The selection of the best EPPS 
scales through the use of the Wherry- 
Doolittle test selection technique re- 
sulted in a coefficient of multiple cor- 
relation of .345 for the experimental 
group. However, when these scales 
were used for the cross-validation 
group, the value of the correlation co- 
efficient degenerated to —.058. 

The results of the study do not nec- 
essarily indicate a lack of usefulness 
for the EPPS, but they do highlight 
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the shortcomings of many of the re- 
search studies which have reported 
success in prediction with the EPPS 


scales. 
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A NOTE ON RELATIONSHIPS}BETWEEN STUDENT 
AND PARENT MINNESOTA TEACHER 
ATTITUDE INVENTORY SCORES 
ROBERT E. CAMPBELL ann JOHN E. HORROCKS 
Ohio State University 


Influences contributing to the for- 
mation of attitudes toward teaching 
and pupil-teacher relationships are 
complex. They are also difficult to 
identify with any degree of compre- 
hensiveness or accuracy. Riccio 
(1959), in a survey and study of 
the literature of teacher attitudes 
identified a number of related vari- 
ables. The variables included per- 
sonality traits, experiences, values, 
and, to some extent, academic ap- 
titude. To the best of the writers’ 
knowledge, an overlooked variable 
has been the role of parents in help- 
ing form the professional attitudes of 
those of their children who become 
teachers. The present study deals with 
possible relationships between profes- 
sional attitudes of teachers-in-training 
and those of their parents. Three ques- 
tions were considered: How similar 
are the attitudes of students and their 
parents? Does the student tend to 
identify attitudinally more with one 
parent than with the other? What is 
the relationship between the attitudes 
of a student’s parents? 


PROCEDURE 


The Minnesota Teacher Attitude Inven- 
tory (MTAI) (Cook, Leeds, & Callis, 1952) 
was administered -to 127 Ohio State Uni- 
versity College of Education students and 
their parents. The students were mainly 
sophomores enrolled in an educational psy- 
chology course. Both students and parents 
volunteered to participate in the study as 
a partial course requirement. The MTAI 
was administered to the students in a 
classroom session at the beginning of the 
course. A month later inventories and an 
explanatory letter were sent to parents. The 
letter stated that the investigators were in- 


terested in parental attitudes toward educa- 
tion. The parents were requested to answer 
their questionnaires separately and to re- 
frain from showing their answers to each 
other. Returned answer sheets were care- 
fully screened for misinterpretation of di- 
rections. 


RESULTS AND Discussion 


Table 1 shows the relationship be- 
tween the MTAI scores of students 
and those of their parents. The data 
suggest three trends: the moderate 
positive (.45) agreement between par- 
ents’ attitudes; the similarity of atti- 
tudes between students and their par- 
ents; and though tenuously, the 
students’ attitudinal identification 
with the parent of the same sex. 
Daughters and mothers present a 
higher relationship (.37) than did 
daughters and fathers (.15).' Simi- 
larly the son-father correlation (.33) 
was higher than the son-mother corre- 
lation (.19). Although these differences 
were not statistically reliable, trends 
in these directions were not unexpected 
in view of similar results reported in 
studies made in other areas of parent- 
child identification. Father-son and 
mother-daughter identifications were 
noted by Gray and Klaus (1956) in 
their study of values, and by Hender- 
son (1958) in his study of interests. 
Lazowick (1955), using an S-R ap- 
proach, has also reported on the nature 
of identification. 

Table 2 presents the means and 
standard deviations for the categories 
of groups used. Although the variance 

*The difference between the correlations 


was not statistically significant at the 05 
level of confidence. 
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TABLE 1 


Propuct-MomMENT CoRRELATIONS BETWEEN 
SrupENTs’ AND Parents’ MTAI Scores 


Group r N 


-285* 127 
.334* 127 
-449* 127 
.330* 39 
.189** 39 
.149°* 

-369* 88 


Students-fathers 
Students-mothers 
Mothers-fathers 
Sons-fathers 
Sons-mothers 
Daughters-fathers 
Daughters-mothers 


* Significant at .01 level of confidence. 
** Significant at .05 level. 


TABLE 2 


MEANS AND STANDARD DEVIATIONS OF 
MTAI Scores ror PARENTS 
AND STUDENTS 


Group N 


All students 
Mothers 
Fathers 
Daughters 
Sons 


is homogeneous for all groups, the 
means are significantly different. Since 
the MTAI may be interpreted on an 
authoritarianism-permissivéness con- 
tinuum (high positive scores represent 
permissiveness in teacher-pupil rela- 
tionships whereas low positive or nega- 
tive scores present authoritarian atti- 
tudes) it is possible to rank daughters 
as most permissive with sons, mothers, 
and fathers following in that order. It 
would appear that an age as well as a 
sex variable may have been operating 
here. 


ROBERT E. CAMPBELL AND JOHN E. HORROCKS 


SUMMARY AND CONCLUSIONS 


Similarities of teaching attitudes of 
127 teachers-in-training and their par- 
ents were investigated by means of the 
Minnesota Teacher Attitude Inven- 
tory. 

The results suggest the following 
conclusions: (a) parents are in rela- 
tively moderate agreement, (b) a fair 
positive relationship exists between 
the attitudes of students and those of 
their parents, (c) students tend to 
identify attitudinally with the parent 
of the same sex, (d) daughters are the 
most permissive in their concept of 
teacher-pupil relationships while sons 
are less so, and (e) mothers and 
fathers tend to be more authoritarian 
in their MTAI responses than do their 
children of either sex. 
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TEST ANXIETY AND THE INTELLECTUAL 
PERFORMANCE OF COLLEGE 
STUDENTS! 


IRWIN G. SARASON?*® 
University of Washington 


A currently strong trend in the se- 
lection of college students is the reli- 
ance on objective tests of intelligence 
and aptitude. Very likely the problems 
associated with an increasing college 
age population and the heightened de- 
mands for admission to colleges are 
going to accelerate this trend. While 
the value of entrance examinations 
and objective measures of intellectual 
performance is obvious, a legitimate 
question would seem to be whether or 
not factors other than ability and ap- 
titude affect the test performance of 
students. If it were the case, for ex- 
ample, that certain personality char- 
acteristics correlated negatively with 
intellectual performance measures, it 
would seem important to take this 
correlation into account in some way 
in evaluating applicants. 

Although the problem of the pos- 
sible confounding of personality with 
intellectual variables has been stated 
specifically in terms of admission to 
college, it is clear that this is but one 
aspect of the more general problem of 
the relationship between intelligence 
and personality. This is so because all 
students in most school systems un- 
dergo systematic intellectual assess- 
ment, and educational decisions fre- 
quently are made on the basis of this 
assessment. 


? This research was supported by a grant 
from the Agnes H. Anderson Research Fund. 

*The writer is indebted to Paul Horst 
and the staff of the Counseling and Testing 
Service of the University of Washington for 
their cooperation in making available these 
data. 


The present research investigated 
the relationship between intellectual 
and personality measures for male and 
female freshman and sophomore col- 
lege students. Thirteen intellectual 
measures were available for each stu- 
dent, of which six were high school 
grade point averages in a variety of 
fields, and seven were objective tests 
included in an entrance battery. Six 
personality variables were correlated 
with each of these 13 intellectual 
measures. Three of these six person- 
ality variables were measures of what 
might be called general anxiety, i.e., 
anxiety as expressed or experienced in 
a variety of situations. One other 
measure tapped the student’s feelings 
of hostility in a variety of situations 
and another measure was designed to 
assess the subject’s level of defensive- 
ness. The sixth personality variable 
studied, and the one which it was felt 
would most strongly relate to intellec- 
tual performance, was test anxiety, 
ie., anxiety experienced specifically 
in testing situations. The prediction 
of significant relationships between 
test anxiety and intellectual perform- 
ance was based on the following for- 
mulation. 

Anxiety responses are aroused by 
personally threatening conditions in 
the environment. These responses in- 
terfere with ongoing task-relevant ac- 
tivity and lead to a lowering of per- 
formance level. Although the exact 
nature of the interfering anxiety re- 
sponses is not known, it seems likely 
that they include both heightened 
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physiological activity (e.g., heart rate, 
palmar sweating) and self-deprecat- 
ing ruminations (e.g., “I can’t pass 
this test,” “I wonder how the other 
students are doing”). While these anx- 
iety responses are assumed to be 
aroused under stressful conditions, 
their evocation would not necessarily 
be expected under neutral conditions 
(Sarason, 1960). 

Within this view of anxiety, anxiety 
scales or questionnaires may be con- 
sidered to measure the likelihood of 
emission of anxiety responses under 
conditions of personal threat. Further- 
more the closer the content of the anx- 
iety scale is to the situation in which 
the subject’s performance is measured 
the better predictor of reaction to 
threat will the anxiety scale be. Since 
the test anxiety scale used in this 
study contains items dealing with the 
subject’s reactions to testing situa- 
tions, it was expected that it would 
most meaningfully relate to perform- 
ance. Because of the importance of 
college aptitude tests to students it 
was predicted that test anxiety would 
correlate negatively with the seven 
entrance examination tests. The cor- 
relations of test anxiety with the six 
high school grade point averages were 
expected to be of a lower order than 
those with the entrance test battery. 
This latter prediction was made be- 
cause of the possibility that while 
highly anxious students might not be 
able to prepare for and cope ade- 
quately with entrance and aptitude 
tests, they well might be able to re- 
duce their anxiety levels over a se- 
mester or school year in course work. 


METHOD 


Subjects. The subjects were 326 male and 
412 female students enrolled in introductory 
psychology and introductory sociology 
courses at the University of Washington. 
The Autobiographical Survey containing 
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the personality measures (Sarason, 1958b) 
was administered during regular class ses- 
sions. The aptitude measures were obtained 
prior to the subjects’ entering the Univer- 
sity of Washington. Thus the personality 
and intellectual measures were not obtained 
contemporaneously. 

Materials. The form of the Autobio- 
graphical Survey used in this research con- 
sists of six true-false scales: Test Anxiety, 
(TA), 17 items; General Anxiety (GA), 18 
items; Lack of Protection (LP), 27 items; 
Hostility (H), 22 items; Need for Achieve- 
ment (NA), 31 items; Defensiveness (D), 
24 items. Descriptions of the scales have 
been presented previously (Sarason 1958b)* 

The 13 intellectual measures employed are 
routinely gathered for entering University 
of Washington students. The measures are: 

1. High school English average 

2. High school mathematics average 

3. High school foreign language average 

4. High school social studies average 

5. High school natural science average 

6. High school electives average 

7. Verbal score on the Guilford-Zimmer- 
man Aptitude Survey 

8. Mechanical Knowledge score on the 
Guilford-Zimmerman Aptitude Survey 

9. Cooperative English Usage score 

10. Cooperative English Spelling score 
11. Cooperative Mathematics I score 
12. Cooperative Social Studies II score 
13. ACE (1948) Q score 


RESULTS 


The results for the male and female 
subjects will be presented first in 
terms of the correlations between the 
intellectual and personality measures 
and then in terms of the intercorrela- 
tions of the Autobiographical Survey 
scores. 

Table 1 contains the correlations 
between the 6 personality and 13 in- 
tellectual variables for the 326 male 
subjects. The table also contains the 
means for the 13 intellectual measures. 
Although GA correlated negatively 
and significantly with 4 intellectual 
measures and NA correlated nega- 
tively and significantly with 5 of these 


*Copies of the Autobiographical Survey 
may be obtained from the author. 
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TABLE 1 
CoRRELATIONS OF Six AUTOBIOGRAPHICAL 
Survey Scores with THIRTEEN 


INTELLECTUAL MEASURES FOR 
326 


| 
ita | D 
| | 


Natural science av- 
erage 

Elective average 

G-Z Verbal 

G-Z Mechanical 
Knowledge 

Cooperative English 


Usage 
Cooperative English 
Spelling 
Cooperative Mathe- 
matics I 
Cooperative Social 
Studies II 
ACE Q score (1948) 


27.18 
21.85 


$s 8 8 


86.61 


16.17 


23.71 
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| 


06 
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| 18.45 
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Note.—r = .133 at .05 level; r = .148 at .01 level. 


measures, it is clear that TA shows 
the most consistent set of correlations. 
For 11 of the 13 intellectual measures 
there were significant negative corre- 
lations with TA. 

A comparable set of correlations for 
412 females (Table 2) also shows TA 
to correlate negatively and signifi- 
cantly with 11 of the 13 intellectual 
measures, although the same 11 vari- 
ables were not significant for both 
males and females. The only other 
personality characteristic found to be 
related to intelligence was H which 
correlated positively with four intel- 
lectual measures. 

For both the male and female 
groups, comparisons were made of the 
correlations obtained between TA and 
high school grade point averages and 
TA and entrance tests. For the pur- 
poses of this comparison the six TA 
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grade point averages were compared 
with six TA entrance test scores. Not 
considered were the male and female 
TA-Mechanical Knowledge correla- 
tions. This omission was based on the 
great differences between male and fe- 
male means on this test. The compari- 
sons made indicated stronger negative 
correlations between TA and the six 
entrance tests than between TA and 
the six high school grade point aver- 
ages. This result was significant for 
both males and females (p < .05 and 
p < .02, respectively). 
Intercorrelations between the six 
Autobiographical Survey measures 
and the means for these measures are 
presented in Tables 3 and 4, for males 
and females, respectively. In general 
the patterns of intercorrelations for 
males and females are quite compara- 
ble. For both sexes the D measure is 


TABLE 2 


CoRRELATIONS OF Six AUTOBIOGRAPHICAL 
Survey Scores THIRTEEN 
INTELLECTUAL MEASURES FOR 
412 FemMaLes 


LP 


01 


Natural science av- 
erage 

Elective average 

G-Z Verbal 

G-Z Mechanical 4.51 
Knowledge 

101.78 


20.47 


17.15 


—05 16.43 


ACE Q score (1948) 


Note.—r = .008 at .05 level; r = .128 at .01 level. 
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TABLE 3 


INTERCORRELATIONS OF S1x AvUTO- 
BIOGRAPHICAL SuRVEY SCORES 


FoR 326 MALzEs 
Scores | TA| GA| LP| H | NA| D Mean 
TA 53 | 35 | 13 | 34 | —50) 4.97 
GA 58 | 23 | 34 | —64) 5.57 
LP 33 | 37 | —56) 6.32 
H 20 | 8.86 
NA —35) 11.57 
D 12.50 


Note.—r = .113 at .06 level; r= .148 at .01 level. 


TABLE 4 
INTERCORRELATIONS OF Srx AvTo- 
BIOGRAPHICAL SURVEY SCORES 
ror 412 FEMALES 


Scores | TA | GA| LP | H | NA; D Mean 
TA 48 | 35 | 08 | 35 | —45) 5.64 
GA 54 | 26 | 31 | —60) 6.40 
LP 33 | 24 | —55) 7.08 
H 14 | —44) 6.96 
NA —31| 10.97 
D 13.42 


Note.—r = .098 at .05 level; r = .128 at .01 level. 


negatively related to each of the other 
Autobiographical Survey measures. 


Discussion 


The present findings demonstrate 
that for both men and women there 
are significant negative relationships 
between anxiety reported by subjects 
as being experienced in testing situa- 
tions (TA) and a variety of measures 
of intellectual performance. These re- 
sults also confirm the findings of a 
previous study (Sarason, 1959) which 
showed (a) TA to be negatively re- 
lated to many of the same intellectual 
variables with which the present study 
was concerned and (b) other person- 
ality measures to be unrelated to in- 
tellectual performance. In the present 
research none of the personality scales, 
except TA, related significantly to 
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performance in any consistent manner 
for both male and female college stu- 
dents. Thus, support was given for 
the hypothesis that the more specific 
the measure of personality (in this 
case, anxiety) is to the situation being 
studied, the more consistently will 
the personality measure relate to per- 
formance (Sarason, 1960). It was hy- 
pothesized also that TA scores would 
be more negatively related to aptitude 
test measures than to the six high 
school grade point averages. The pres- 
ent study suggests that this well may 
be the case. The Need for Achievement 
scale might have been expected to re- 
late negatively to intellectual per- 
formance since its items appear to 
reflect strong need and anxiety to 
achieve. There was an indication that 
this might be true for male students 
but not for female students. 

Two problems then seem to be posed 
by the present findings of consistent 
negative relationships between TA 
and 13 intellectual measures and also 
of greater strength of those negative 
relationships involving aptitude tests 
as compared with high school grade 
point averages. First, what factors 
contribute to the negative relation- 
ships obtained? Second, why is test 
anxiety less of a detrimental factor 
for course grades than for aptitude 
tests? 

With respect to the first question, 
it would seem prudent at present to 
interpret the present results as simply 
indications of poorer intellectual per- 
formance for high than for low test 
anxious students. If this is due to in- 
tellectual differences among subjects 
differing in TA scores, then it may be 
the case that high test anxiety scores 
simply reflect realistic concern over 
intellectual ability. If, however, stu- 
dents differing in test anxiety do not 
differ in intellectual ability then it 
would be worthwhile to determine 
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which psychological aspects of testing 
and academic situations arouse anx- 
iety responses in high TA scorers and 
whether or not the conditions of either 
the testing situation or of anxious stu- 
dents can be modified in such a way 
as to raise anxious students’ perform- 
ance level. Results of experimental 
studies indicate that under neutral or 
reassuring conditions high and low 
anxious subjects do not differ in their 
performance levels but that under 
stress they do differ in the same direc- 
tion as obtained in the present re- 
search (Sarason, 1958a; Sarason & 
Palola, 1960). 

In view of this evidence it seems 
possible that the answer to the second 
question posed, that of the explana- 
tion of the stronger correlations with 
TA of aptitude test scores than of 
grade point averages, may be in the 
opportunity of the student in class to 
be reassured and to reassure himself. 
In the course of a quarter or a semes- 
ter the anxious student can come to 
know his instructors and peers, as well 
as to increase his familiarity with and 
to overlearn the material to be learned. 
These possibilities are largely absent 
in group aptitude testing. Greater em- 
phasis in future research on the inter- 
personal aspects of learning and per- 
formance situations should make 
clearer the implications of these find- 
ings. 

Theoretically, the present study has 
supported the interfering response in- 
terpretation of anxiety discussed ear- 
lier. These results are consistent with 
a conception of the TA as an index 
of proneness to performance disrup- 
tion under personally threatening con- 
ditions. Practically, these findings 
strongly suggest the need to take ac- 
count of the test anxiety factor in in- 
terpreting. students’ intellectual per- 
formance. 

One factor of importance in the 
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present study concerns that of the 
population of subjects used. While TA 
correlated consistently with the in- 
tellectual measures used, in most cases 
the magnitude of the r’s was not great. 
It seems possible that the restriction 
in range of intelligence inherent in a 
college sample may have worked 
against obtaining even more signifi- 
cant r’s than were found in the pres- 
ent research. Future research with 
high school level subjects and a 
greater range in intellectual ability 
should, if this is correct, lead to larger 
r’s than were obtained in this study.‘ 

The intercorrelations of the Auto- 
biographical Survey scales show es- 
sentially the same patterns for the 
women as for the men. Interestingly 
the present intercorrelations for col- 
lege students are quite similar to the 
intercorrelations previously reported 
for hospitalized psychiatric patients 
(Sarason, 1958b). The only sizable 
differences in the correlations for the 
two subject populations are the r’s in- 
volving the Hostility scale. For the 
patient population this scale corre- 
lated much more strongly with the 
other Autobiographical Survey meas- 
ures than was true in the case of col- 
lege students. 


SUMMARY 


The present study correlated scores 
on six personality scales of the Auto- 
biographical Survey with scores on 
seven aptitude test scores and six high 
school grade point averages. The sub- 
jects were 326 male and 412 female 
college students. 

The results demonstrated that of 
the six personality scales only the 
Test Anxiety scale (TA) related con- 
sistently to the 13 intellectual meas- 
ures. For both males and females, TA 

*A recently completed study conducted 
by the writer with high school students as 


subjects provides support for this expecta- 
tion. 
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correlated negatively and significantly 
with 11 of the 13 intellectual meas- 
ures. There were indications, however, 
that the negative correlations with TA 
were stronger for aptitude test scores 
than for grade point averages. These 
results were interpreted as supporting 
the hypotheses that: (a) the closer 
the content of anxiety scales’ items is 
to the situation in which subjects are 
to perform the more predictive will 
the items be, (b) TA scores reflect the 
proneness of subjects to emit inter- 
fering anxiety responses under con- 
ditions of threat in a testing situation. 

Comparisons of the intercorrelation 
matrices of the six Autobiographical 
Survey scales for males and females 
showed quite similar patterns. The 
Autobiographical Survey intercorre- 
lations for male and female college 
students were found to be similar to 


IRWIN G. SARASON 


those previously reported for a neuro- 
psychiatric population. 
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EFFECTS OF MEMORY, EVALUATIVE, AND CREATIVE 
READING SETS ON TEST PERFORMANCE 
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For several years the senior author 
(Torrance, 1950, 1951) has been ex- 
perimenting with a variety of proce- 
dures which might increase the like- 
lihood that students in mental hygiene 
courses will make use of scientifically 
developed principles of mental health 
in their work. Several recent experi- 
ments (Torrance, 1960) have been mo- 
tivated by the belief that the failure 
of educators to make more effective 
use of such principles results from the 
fact that they have not been educated 
to think creatively or imaginatively 
about course materials. It soon be- 
came evident in this experimentation 
that many, perhaps most, students did 
not have the expectation that research 
results or scientifically developed 
principles could be applied imagina- 
tively in solving many of their per- 
sonal and professional problems. At 
best they were looking for specific 
prescriptions. 

The rather shocking realization that 
students enrolled in a graduate course 
in mental hygiene did not even expect 
to be able to use the knowledge they 
were acquiring led the senior author 
to try to develop ways of establishing 
appropriate “sets.” In one study (Tor- 
rance, 1960) random halves of two 
classes were asked to read research 
articles creatively or critically. Those 
who read and reported on the articles 
creatively were later more successful 
in developing a new idea of their own 
and in responding to a test of creative 
applications. These results stimulated 
the present study, an extension of 
“sets” to textbook readings. 

The effect of appropriate prepara- 


tory sets in facilitating learning has 
been well-established (Johnson, 1955, 
p. 77). Usually a person’s activities 
are integrated and he does not react 
to every event that stimulates his 
sense organs. He is selective not only 
of what he responds to but also of 
what is produced and of the mental 
operation used in responding. 

In designing the present experiment, 
the authors have been guided by Guil- 
ford’s (1959) model of human intel- 
lect. Particular attention has been 
paid to his five mental operations: 
cognition, memory, convergent think- 
ing, divergent thinking, and evalua- 
tion. By “cognition,” Guilford means 
the operations involved in recognizing 
facts, discovery, and _ rediscovery. 
Memory involves the retention of 
what has been cognized. The two kinds 
of productive thinking, convergent 
and divergent, include the generation 
of new information from known in- 
formation and remembered informa- 
tion. Convergent thinking leads to a 
conventional solution; divergent 
thinking leads to new and untested 
solutions. Evaluation refers to deci- 
sions as to the goodness, suitability, 
or adequacy of what we know, re- 
member, or produce through conver- 
gent or divergent thinking. In this ex- 
periment a deliberate attempt was 
made to establish sets which would re- 
sult in utilization of these operations. 


MeErTHOD 
Subjects 


The subjects were 115 students in a 
graduate level course in Personality Devel- 
opment and Mental Hygiene at the Uni- 
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versity of Minnesota during the summer of 
1960. Seventy-six were males and 39 were fe- 
males. Most subjects had had professional 
experience in one or more phases of educa- 
tion; 5 were public health nurses and 10 had 
had no professional experience. The average 
number of years of experience was 6.6 years 
and the range was from 0 to 30 years. 

Subjects were assigned alphabetically to 
one or another of three groups (1, 2, or 3). 
The group number signified the reading set 
to be followed during a period of one week. 
This method of assignment to groups re- 
sulted in no biases due to sex, professional 
field, or number of years of experience. Some 
members of the original sample canceled 
the course or otherwise failed to participate 
in certain phases of the study, making the 
size of the three groups slightly unequal. 


Description of the Reading Sets 


The instructions of the memory, evalua- 
tion, and creative application sets were 
given to the subjects as follows: 

Set A—Memory. When you read, it is 
important that you read with a set or in- 
tent to remember everything you read. In 
reading your assignments for the next 
three days, practice using this set. Try to 
make your mind act as a “sponge” and 
“soak up” as much as possible of whatever 
you read. Use whatever devices you like 
for trying to remember what you read. It 
may take some practice before you are 
really successful in assuming this set, but 
do not be discouraged. By the third day, 
you should find it easy to assume this set. 

Set B—Evaluation. When you read, it 
is important that you think about what 
you are reading and evaluate critically 
the content. In reading your assignments 
for the next three days, practice using this 
set. As you read, ask: “Is what the author 
is saying really true? Does what he is say- 
ing agree with what I have experienced? 
Does what he is saying agree with what is 
known through scientific research?” Try 
to find defects in the reasoning of the au- 
thor, in his factual presentation, and the 
like. Do not be concerned about errors in 
grammar, punctuation, sentence structure, 
and the like. It may take some practice 
... (ete., same as above). 

Set C—Creative Application. When you 
read, it is important that you think about 
the many uses of the information which 
you are reading. It is especially important 
that you think of the various ways in 
which the information could be used in 
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your personal and professional life. In 
reading, do not just ask, “What is the au- 
thor saying?” Also ask, “How can I use 
what the author is saying?” Do not stop 
with just one use. Think of as many uses 
as you can of the important ideas pre- 
sented. Jot down some of these uses for 
future reference. It may take some prac- 
tice .. . (ete.). 


Experimental Procedure 


At the beginning of the first experimental 
week, subjects received reading-set instruc- 
tions in accord with the group to which 
they had been assigned. Each subject re- 
ceived only one set during any one week. 
Thus, during the first week subjects in 
Group 1 received Set A; those in Group 2, 
Set B; those in Group 3, Set C; and so on 
—sets being shifted from group to group 
over the 3 weeks. 

Each week each subject was provided an 
Evaluation of Reading Set Form upon which 
he was asked to report the degree to which 
he was able to maintain the assigned set, 
the degree of difficulty experienced in using 
that set, the number of minutes spent in 
completing the reading assignment for the 
last day of the week, and semantic reactions 
to the experience. Osgood’s (1952) semantic 
differential technique was used in obtaining 
the latter. The following polar-adjective 
pairs were used with seven-point scales: 
good-bad, pleasurable-painful, labored-easy, 
wise-foolish, active-passive, fast-slow, cau- 
tious-rash, new-old, and interesting-boring. 

Each Friday subjects were administered 
a 20-minute test which covered the reading 
assignment for that day. This test included 
items which were defined as: cognitive (mul- 
tiple-choice), memory (fill-in-the-blanks), 
divergent thinking (extensions from given 
facts, uses of facts, questions, hypotheses, 
etc.), and evaluation (decision making situ- 
ations and justifications of decisions). It was 
at this time that the Evaluation of Reading 
Set Forms were collected. 

Part-scores based on performance on 
these subtests were determined for all sub- 
jects and were reported to each subject at 
the beginning of the next week. 

The following are samples of items pre- 
sumed to involve cognition: 

According to a recent study by two New 
York scientists involving tests of physical 
fitness, which of the following findings was 
obtained? 

—American children superior to Euro- 

pean children. 
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——European children superior to Ameri- 
can children. 

——American children superior to Rus- 
sian children. 

——Russian children superior to Ameri- 
can children. 

——American children superior to Aus- 
tralian children. 

Which of the following is least likely to 

contribute to the prevention of paranoid 

behavior? 

——Training in straight thinking. 

——Fostering attitudes of good sports- 
manship. 

—— Finding available objects to blame. 

——Developing self-ennobling attitudes 
toward sex. 

—— Facilitating social interaction. 

The following are examples of the mem- 

ory items: 

According to Wang, the primary factor 

in personality maladjustment is either 
or of the child’s 
need for parental affection and sympa- 
thetic handling. 

The experimenter who used a mechani- 
eal mother in laboratory studies at the 
University of Wisconsin is—————— . 
The following illustrate the creative-ap- 

plication problems: 

If hunger needs are not satisfied during 
childhood, how is this likely to affect per- 
sonality development? Assume that the 
child has been given enough food for sub- 
sistence but rarely enough to satisfy hun- 
ger. What kind of adult personality would 
you expect? 

Research has shown that lower socio- 
economic individuals have limited ability 
to delay gratification. List as many ways 
as you can think of for using this infor- 
mation in educational situations to pro- 
mote healthy personality adjustment. 

The following is a sample of the evalua- 
tive problems: 

Assume that you are a member of the 
controlling board of the Minneapolis Gen- 
eral Hospital which is considering the es- 
tablishment of a children’s psychiatric 
unit for mentally disturbed children. The 
treatment program will be quite similar 
to that described in the text for the Chil- 
dren’s Psychiatric Unit at the University 
of Michigan and it is estimated that the 
cost of treatment per child will be $6,500. 
In deciding this issue, how would you 
vote? 

—Yes ——No 
How would you justify your vote? Cite as 
many sound reasons as you can. 
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Analysis of Data 


Analyses of variance were computed, 
comparing the effects of the reading sets of 
each of the different kinds of test perform- 
ance (cognitive, memory, divergent think- 
ing, and evaluation) for each of the 3 weeks. 
The evaluative-type item, unfortunately, 
was used in only one of the tests. Analyses 
of variance were computed for success in 
maintaining set, difficulty in maintaining 
set, number of minutes spent in completing 
the reading assignment, and each of the nine 
semantic differential scales for the three sets. 
Analyses of variance were also computed 
on Test Scores X Weeks and for Weeks 
Groups in order to determine relative per- 
formance by weeks and by groups. 


RESULTS 


The data comparing scores on the 
memory-type items are presented in 
Table 1. It will be noted that only 
during the third week were the differ- 
ential effects of the three sets statisti- 
eally significant at the .05 level of sig- 
nificance or better, the memory set 
producing the highest score. The pre- 
dicted trend occurred during the sec- 
ond week but was not statistically 
significant. 

Table 2 presents the means and 
analysis of variance data for scores 
on the Creative-Applications or diver- 
gent-thinking items. It will be noted 
that the differential effects are rather 
strong for each of the 3 weeks and that 
in all cases those working under the 
creative set achieved the highest mean 
score. Those using the evaluative set 
consistently achieved the next highest 
mean score. On the third test all three 
groups tended to achieve higher scores 
than during the earlier weeks. This is 
probably due to the accumulative ef- 
fects of the course which emphasized 
creative thinking concerning the 
course content. 

The data concerning performance 
on the evaluative (decision making) 
items for the second week are shown 
in Table 3. The differential effects are 
statistically significant, with the eval- 
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TABLE 1 


Comparison oF Errects oF THREE READING Sets ON PERFORMANCE IN 
Memory Type Tests (ComMPpLeTION) ror Eacu or Tuoree WEEKS 


= 


Reading set 


M Between SS 


Within SS F ratio 


Memory 
Evaluative 
Creative 


Memory 
Evaluative 
Creative 


Sees $865 


Memory 
Evaluative 
Creative 


NOW NON 
SRS 


3.9259 195.7898 


*p< 6. 


TABLE 2 


ComPaRIsON oF Errects oF THREE REeapING Sets ON PERFORMANCE IN 
CREATIVE-APPLICATION Types OF Tests ror Eacu or THREE WEEKS 


Reading set N M 


Between SS Within SS F ratio 


_ Memory 
Evaluative 
Creative 


Memory 
Evaluative 
Creative 


$85 


Memory 
Evaluative 
Creative 


4.45 
4.94 
7.96 
6.88 
8.37 
8.60 


72.4602 643.6542 12.496** 
76.1493** 


13.7613** 


< Ol. 


TABLE 3 


ComPaRISON oF Errects or THREE 
ReapiInG Sets ON PERFORMANCE IN 
Evatvuative (Dectston 

Type or Test ror Seconp 


— F ratio 


Be- 
Reading set tween 
SS 


| 5.88 
41 | 7.34 
37 | 6.70 


Memory 
Evaluative 
Creative 


39.1292) 368.4517 


Ol. 


uative, creative, and memory sets 
ranking in that order. Although this 
type of item was used at other times 
during the course, the only test in the 


experiment containing such an item 
was the one for the second week. 

Table 4 presents the data for the 
cognitive items for each of the three 
tests. It had been anticipated that 
subjects using the memory set would 
perform best on these items. This oc- 
curred only during the second week. 
The differential effects were also sig- 
nificant for the last test, but subjects 
working under the evaluative set 
achieved the highest mean score. It 
will also be noted that subjects using 
the evaluative set also had a slight 
edge during the first week but the dif- 
ferential effects were not statistically 
significant. 


of 
210 
‘a 
= 
First 
2.22 
| 
Second 
9.8958 420. 2684 2.543 
12.7482 405.2544 3.429* 
— 
First 
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Second 
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TABLE 4 


ComPaRIsoN oF Errects or THree Reapinc Sets oN PERFORMANCE ON COGNITIVE 
Type or Tests ror Eacu or WEEks 


Reading set 


ce Between SS Within SS F ratio 


Memory 
Evaluative 
Creative 


Memory 
Evaluative 
Creative 


S885 


Memory 
Evaluative 
Creative 


SRS 


E88 


1.9062 236.9290 0.893 


200.9297 5.537** 


*p< 6. 
"p< 


Test Performance by Group and by 
Week 


Analyses of variance were also per- 
formed to study differential test per- 
formance by group and by week. It 
was found that the effects for groups 
are not statistically significant. The 
effects for weeks are statistically sig- 
nificant, total mean scores being 
higher for the third week than for 
the first two weeks. Thus, there is no 
evidence of the superiority of any 
one sequence of sets over the others. 


Other Reactions to Assigned Sets 


Failure to maintain an assigned set 
is obviously an important considera- 
tion in determining the effectiveness 
of a set in producing the desired re- 
sults. Two questions on the Eval- 
uation of Reading Set Form were 
designed to supply information con- 
cerning this problem. Responses con- 
cerning the completeness with which 
the set was maintained ranged from 
“not at all” (Point 1) to “almost all of 
the time” (Point 7). Responses con- 
cerning difficulty in learning to use the 
set ranged from “impossible” (Point 
1) to “extremely easy” (Point 6). 
Weights were assigned arbitrarily. 


Means and F ratios are presented in 
Table 5 for the variables: success in 
using set, difficulty in using set, num- 
ber of minutes spent in reading as- 
signment for day, and for each of the 
semantic scales. 

From Table 5 it will be noted that 
the sets had significant differential ef- 
fects on both success and difficulty in 
using set but not on time spent on 
reading assignment. The memory set 
seems to have been the most success- 
fully and easily maintained and the 
creative set seems to have been the 
least successfully and easily main- 
tained. Subjects estimated that they 
maintained the memory set on the 
average about two-thirds of the time, 
finding it between “fairly difficult” 
and “fairly easy” to maintain. They 
reported that they were successful in 
using the creative set less than half 
of the time. 

From Table 5 it will also be noted 
that on only four of the nine polar- 
adjective pairs are reactions signifi- 
cantly different among the three con- 
ditions of reading set. Those pairs are: 
fast-slow, cautious-rash, new-old, and 
interesting-boring. Subjects using the 
memory set tended towards fast, cau- 
tious, old, and boring. Those using the 
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TABLE 5 
Comparison or Errects or THree Reapine Ses on Success 1n Usine Set, 
Dirricutty 1n Usine Set, Time Spent on Reapine ASSIGNMENT, AND 
Semantic Reaction ror THree Weexs ComMBINED 


Memory 


Evaluative 


N 


Success in using set 106 
Difficulty in keeping as- 106 
signed set 
Number of minutes 104 
spent on reading as- 
signment 
Good-Bad 105 
Pleasurable-Painful 103 
Labored-Easy 103 
Wise-Foolish 103 
Active-Passive 103 
Fast-Slow 102 
Cautious-Rash 102 
New-Old 103 
Interesting-Boring 103 


*p< 
"p< 


evaluative set tended towards slow 
and held to a middle position on the 


other pairs. Under the creative set the 
direction was towards rash, new, and 
interesting. 

In order to determine whether or 
not there are differential effects with 
the passage of time, a comparison was 
made by weeks on the variables listed 
in Table 5. Only four of the variables 
gave rise to changes with time as data 
of analyses of variance showed. Dur- 
ing the second week subjects tended 
to report greater success in maintain- 
ing the sets than during the other 
weeks. They also tended to spend 
somewhat more time on the reading 
assignment during this particular 
week. Finally, there was a consistent 
change towards fast and boring over 
the 3 weeks. Of these four variables 
the time spent on the reading assign- 
ment was significant at .01 < F < .05. 
The other three were significant at 
<.01 level. 


Discussion 

From the data which have been pre- 
sented it seems clear that instructing 
mature students to assume various 
reading sets will have differential ef- 
fects on the kind of goals achieved. 
This would seem to be especially true 
if creative thinking and evaluative 
thinking are desired as outcomes. Al- 
though the students in the present 
study were able to maintain the crea- 
tive and evaluative sets only about 
half of the time, attempts to maintain 
these sets had measurable effects on 
performance on tests covering the 
reading assignment. 

Since most examinations given to 
large classes are of the cognitive (mul- 
tiple-choice) type, it is worthwhile to 
note that the memory set does not uni- 
formly result in the best performance 
on this type of test. In fact, in only 
one of the 3 weeks does the memory 
set result in superiority on the cogni- 
tive type of test. The evaluative set 


d | 
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tended to result in better mean scores 
during the other 2 weeks. 

Because of the differential effects 
of the three sets on the four types of 
tests, one might also infer that in- 
structors need to develop tests de- 
signed to measure different kinds of 
achievement, depending upon course 
objectives. If instructors expect their 
students to think imaginatively and 
evaluatively about what they learn, 
it is extremely doubtful that such 
goals are adequately assessed by the 
traditional multiple-choice and com- 
pletion items. 

The data indicate that students are 
more successful in maintaining the 
memory set and find it easier to main- 
tain this set than the creative and 
evaluative sets. This probably results 
from the fact that they have had much 
more practice with this set than with 
the others. This hypothesis receives 
some support from reactions to the 
semantic differential. For example, 
suhjects under the memory set tend 
to fate the experience in the direction 
of cautious, old, and boring. It is likely 
that, with practice, students will find 
it easier to maintain creative and 
evaluative sets and that better results 
will be achieved than those reported 
herein. 

Finally, in interpreting the results, 
it should be recognized that the sub- 
jects are graduate students and that 
almost all of them have had rather 
extensive experience in various types 
of educational work. 


SuMMARY 


This experiment was designed to 
test the differential effects of memory, 
evaluative, and creative “sets” in 
reading assignments in a graduate 
level course on Personality Develop- 
ment and Mental Hygiene. The 115 
subjects were arranged alphabetically 
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and assigned to one of three groups. 
Each group was assigned in turn each 
of the three reading sets to be used in 
the assigned readings for a week. At 
the end of each week, a 20-minute test 
containing cognitive, memory, crea- 
tive thinking, and evaluative items 
was administered. Subjects also esti- 
mated their degree of success in main- 
taining the set, difficulty in maintain- 
ing it, the number of minutes spent 
on the Friday’s reading assignment, 
and their semantic reactions to nine 
polar-adjective pairs. 

Analyses of variance indicated that 
the three sets produced differential ef- 
fects on most of the tests for all 3 
weeks. Those operating under the 
creative set, achieved the highest 
mean each week on the creative appli- 
cations. Those operating under the 
evaluative set achieved the highest 
mean on the evaluative or decision 
making problem when such a problem 
was given (second week only). Those 
operating under the memory set 
achieved the highest mean on the 
memory tests (completion) only dur- 
ing the third week. The differential 
effects on the memory items were not 
statistically significant during the 
other weeks. Differential effects on 
the cognitive items were not consis- 
tent. Those operating under the mem- 
ory set achieved the highest mean 
during the second week and those un- 
der the evaluative set led during the 
third week. 

Subjects reported that they found 
it easier to maintain the memory set 
and rated themselves as more success- 
ful in maintaining this set than the 
other two. There were no differential 
effects on the estimated length of time 
spent on the reading assignment. On 
the semantic differential, subjects un- 
der the memory set tended towards 
fast, cautious, old, and boring. Those 
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under the evaluative set tended to- 
wards slow and those under the crea- 
tive set, towards rash, new, and in- 
teresting. 

During the second week subjects 
generally reported greater success in 
maintaining sets than during the first 
and third weeks. On the semantic 
evaluations, there was a consistent and 
significant drift towards fast and bor- 
ing. The three original groups did not 
differ significantly, giving no evidence 
of the superiority of any one sequence 
of sets over the others. 
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THE EFFECTS OF VISUAL DISCRIMINATION 
PRETRAINING WITH WORD AND LETTER 
STIMULI ON LEARNING TO READ A 
WORD LIST IN KINDERGARTEN 
CHILDREN 


SIEGMAR MUEHL* 
Iowa Child Welfare Resecrch Station 


Ability to discriminate among word 
forms is a prerequisite to associating 
these same stimuli with appropriate 
word names. 

An earlier study by the author 
(Muehl, 1960) using kindergarten 
subjects, showed that visual discrimi- 
nation pretraining with the same 
words to be read in a subsequent word 
list facilitated learning the word list 
when compared to discrimination pre- 
training with different words or pre- 
training with geometric forms. These 
results suggested the possibility that 
specific observing responses were 
learned to the stimuli in the same- 
word pretraining; and, that these re- 
sponses transferred positively to dis- 
criminating among the words in the 
vocabulary task. 

The purpose of the present experi- 
mental study was to determine, more 
specifically, the stimuli relevant to 
visual discrimination among word 
forms for beginning readers. 

One assumption tested is that the 
child learns to discriminate among 
words on the basis of the shape or 
form of the whole word. A second as- 
sumption tested is that the child dis- 
criminates among words on the basis 
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of attending to some part of the word, 
such as a particular letter, part of a 
letter, or letter grouping. 

The assumption that word shape is 
the basis for visual discrimination 
among words, although lacking objec- 
tive confirmation (Davidson, 1931), 
appears to be supported by educators, 
judging by the widespread practice of 
the so-called “whole-word” method of 
introducing reading in the first grade. 
According to Gray (1956, pp. 83-84), 
this method is based on the assump- 
tion that, “...individual words are 
the basic units of thought and recog- 
nition,” or, that, “...each word has a 
characteristic form by which it can 
be remembered.” Although words are 
the basic meaning units in our lan- 
guage, it does not follow they are also 
the basic units of visual recognition. 

The assumption that the beginning 
reader discriminates among words on 
the basis of letters finds support in 
two different areas of educational re- 
search. Comparing intralist errors in 
word recognition tasks for children 
ranging from 4 to 6 years in age, 
Gates and Boeker (1923), Meek 
(1925), and Muehl (1960) found that 
letter similarities were the apparent 
source of confusion among words. Us- 
ing a camera to record eye move- 
ments, Buswell (1922) showed that 
beginning readers made many eye fix- 
ations associated with details within 
a word in the early reading stages. 

The results of psychological re- 
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search bearing on the related question 
of part vs. whole discrimination are 
inconclusive. A review of this research 
with children, reported by Gibson and 
Olum (1960), showed that the re- 
search results were conflicting and ap- 
peared to depend on the stimulus ma- 
terial used. However, none of the 
studies reviewed had used word stim- 
uli. 


METHOD 


Pretraining Task 


To test these alternative assumptions, 
three groups of kindergarten children were 
differentiated Dy the kinds of visual dis- 
crimination or matching pretraining each 
received prior to learning to read a word 
list. The pretraining paradigm is shown in 
Table 1. The nonsense word sets for Groups 
RR and IR were designed so that either 
word shape (stimulus configuration deter- 
mined by all the letters) or letter differences 
among the three words in each set could 
serve as relevant stimuli for discriminating 
among the words. LD pretraining was in- 
cluded to control for the effect of learning 
to discriminate among the relevant letters, 
as such, on later word list performance. 

Each of the pretraining sets (RR, IR, and 
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LD) was presented in four orders. Two of 
these orders presented consecutively con- 
stituted a two-trial block. The words and 
letters appearing in the sets were typed in 
lower case with a primary-style typewriter. 
The word stimuli were approximately 11 
mm. in length and 5 mm. in height to in- 
clude the tallest letters. These word and 
letter stimuli were mounted in the center 
of 242” xX 2%” white cards. Accompanying 
each two-trial block of stimulus items in the 
three sets were two, 2%” X 8”, multiple- 
choice response cards. The three words, or 
three letters, appearing on the stimulus 
cards in each set also appeared on these re- 
sponse cards, in two different position or- 
ders. The apparatus for presenting the 
stimulus and response cards has been de- 
scribed in detail elsewhere (Muehl, 1960). 
Briefly, it consisted of a tray for presenting 
the stimulus cards, a hinged window below 
the tray for exposing the response card, and 
three buttons located on a panel immedi- 
ately in front of each of the three words ap- 
pearing on the exposed response card. If 
the subject selected the correct. response 
word and pressed the button under it a 
door chime would ring. The experimenter 
controlled the appropriate button-chime 
connection by means of a selector switch 
from behind the apparatus. 

All subjects received eight matching 
trials. A presentation consisted of the ex- 
perimenter’s placing a stimulus card on the 


TABLE 1 
ParapigM To ILLUSTRATE EXPERIMENTAL DeEsIGN 


Reading Task 


Responses 


All Groups 


matching 


same as above 


same as above, plus ori- 
enting instructions 
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card-holder tray for approximately 2 sec- 
onds. The subject was told to look carefully 
at the card. The experimenter then removed 
the card and the subject was instructed to 
open the response card window, find the 
same stimulus item on the response card 
that had appeared on the stimulus card, 
and then press the button directly under it 
to find if his choice was correct. If the sub- 
ject pushed the incorrect button on any 
one of the first three presentations, the ex- 
perimenter showed the stimulus item again 
and had the subject find the correct response 
item and ring the bell. On subsequent pres- 
entations, if the subject responded incor- 
rectly or if the subject did not respond in 
approximately 5 seconds, the experimenter 
closed the response card window and pre- 
sented the next stimulus card. The experi- 
menter alternated response cards with each 
two-trial block. No rewards were given in 
pretraining other than provided by the 
chime, and verbal encouragement from the 
experimenter. 


Reading Task 


The reading or transfer task was de- 
signed to make both word-shape and letter 
differences among the Group RR pretrain- 
ing words relevant stimuli for discriminat- 
ing among the reading task words. For 
Group IR pretraining words, the reading 
task was designed to make letter differences 
alone relevant to discriminating among the 
reading task words. This reading task para- 
digm is shown in Table 1. 

Set RR nonsense words were paired with 
%” X %” color patches. The typed words 
and color patches were mounted on 34%” X 
6%” white, plastic cards for paired-associate 
presentation. 

The apparatus used for the reading task 
was a Hunter Card Master. The Card Mas- 
ter is designed to present automatically the 
above plastic cards before a 3” < 542” aper- 
ture. By means of a timer unit and two metal 
shutters, the aperture opening can be con- 
trolled to secure the following exposure in- 
tervals: an anticipation interval to expose 
the stimulus word alone; a joint presenta- 
tion interval to expose the word-color pair; 
and a between-item interval when the aper- 
ture is completely covered, during which 
time the next stimulus-response pair is 
brought into position behind the shuttero. 

Four orders of the three word-color pairs 
appeared in the reading task. Two of these 
orders presented consecutively constituted 
a two-trial block. A blank card appeared at 
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the end of each two-trial block. The follow- 
ing presentation intervals were used: an- 
ticipation interval—3 seconds; joint pres- 
entation interval—3 seconds; and a 
between-item interval—2 seconds. All sub- 
jects continued in the reading task to a 
learning criterion of 11 of 12 correct antici- 
pations in two successive two-trial blocks, 
or for a maximum of 20 trials. Subjects 
reaching a learning criterion before 20 trials 
were given perfect scores on the remaining 
trials. 

The reading task was administered im- 
mediately following pretraining. At the be- 
ginning of the reading task, the subject was 
shown each color patch with the word con- 
cealed and told the appropriate color name. 
The three color patches were then presented 
a second time and the subject asked to name 
the colors. If the subject could not give the 
correct color names he was eliminated. Fol 
lowing the color naming, a familiarization 
trial with the three word-color pairs was 
given. The subject was told with each paired 
presentation that the word and color went 
together. After presenting the third pair the 
subject was instructed that the game was to 
guess the color that went with each word 
before the color appeared. The learning 
trials began following these instructions for 
subjects in Groups RR, IR, and for half the 
LD group. Since the LD group would not 
see the relevant letters as parts of words 
during pretraining, subjects in this group, 
presumably, would need to learn where to 
look for these letters in the reading task 
words, as well as learn to discriminate the 
relevant letter from the rest of the word. 
To control for learning where to look, the 
other half of the LD group, designated 
LD-O, was given special orienting instruc- 
tions at the beginning of the reading task. 
At the end of the familiarization trial, the 
experimenter told subjects in the LD-O 
group always to look at the beginning of 
the word and this would help them learn 
the right color. At the same time the ex- 
perimenter pointed to the beginning of the 
word. These instructions were repeated twice 
more, at the end of trial Blocks 1-2 and 3-4. 

To secure attention to the stimulus words 
as they appeared in the aperture during the 
reading task, the experimenter frequently 
pointed to the window with a pencil on 
each presentation. If the subject did not re- 
spond during the anticipation interval, the 
experimenter encouraged the subject to try 
to guess . color after looking carefuly at 
each word. 

If a correct response occurred in the an- 
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ticipation interval, the subject moved a 
bead across a counting frame. If an incor- 
rect response or omission occurred, the ex- 
perimenter named the correct color in the 
joint presentation interval. The bead in- 
centive and occasional verbal encourage- 
ment from the experimenter served as re- 
wards in the reading task. 


Subjects 


The subjects were 60 children from three 
kindergartens in the public schools in Iowa 
City. The subjects were randomly assigned 
to the three pretraining groups: 15 each to 
Groups RR and IR, 30 to Group LD. Three 
additional subjects were eliminated from 
the study because they could not respond 
correctly in naming the color patches. Sub- 
jects ranged in age from 61 to 72 months. 
The mean age was 66.7 months. There were 
no significant age differences among the 
three pretraining groups, either in varia- 
bility, or mean age. Testing was done in the 


TABLE 2 
Mean NumBer or Correct REsPoNsES 
DURING VISUAL DISCRIMINATION 
PRETRAINING 


Blocks of two trials 


Group 
34 | 7-8 Total 


WD* | 4.27 | 4.60 | 5.03 | 5.10 | 19.00 
LD 5.47 | 5.23 5.43 | 5.57 | 21.70 


* Groups RR and IR combined. 


TABLE 3 
ANALYSIS OF VARIANCE OF CORRECT 
Responses 1N Eraut LEARNING 
Triats For Groups WD anp 
LD 1n Visvat D1scrim- 
INATION PRETRAINING 


(Two-trial blocks) 


Source af | MS | F > 

Between-Subjects 59 

Groups (B) 1 |27.34)11.02)< .005 

Error (b) 58 | 2.48 
Within-Subjects 180 

Trials (A) 3 | 3.19) 4.25)}<.01 

AXB 3 | 1.98) 2.64/<.10 

Error (w) 174 .75 

Total 239 
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public schools during November and De- 
cember. 


RESULTS 


Pretraining 


Comparing Groups RR and IR, the 
mean number of correct responses 
were 19.40 and 18.60, respectively ; the 
standard deviations 4.22 and 3.52, re- 
spectively. A Lindquist (1956) Type 
I analysis of variance of the pretrain- 
ing scores showed no reliable perform- 
ance differences between these groups. 
The trials effect for the combined 
groups was significant (p < .01), indi- 
cating a learning effect. Since there 
were no performance differences be- 
tween Groups RR and IR in pretrain- 
ing these two word discrimination 
groups were combined and designated 
as Group WD. Comparing Groups 
WD and LD in pretraining, the mean 
number of correct responses were 19.00 
and 21.70, respectively; the standard 
deviations 3.91 and 1.98, respectively. 
Hartley’s test (Walker & Lev, 1953, 
pp. 193-194) for homogeneity of vari- 
ance between the groups was signifi- 
cant (p < .01), leading to the rejec- 
tion of the null hypothesis. The mean 
numbers of correct responses for 
Groups WD and LD during pretrain- 
ing are presented in Table 2. A sum- 
mary of a Lindquist (1956) Type I 
analysis of variance for these data is 
presented in Table 3. As shown in the 
summary, differences in group means 
and the trials effect were significant 
(p = .025, to adjust for effects of het- 
erogeneity of variance). A further 
analysis of simple effects showed that 
the trials or learning effect was en- 
tirely accounted for within the WD 
group (p < .01). 


Reading Task 


Comparing Groups RR and IR, the 
mean number of correct responses 
were 33.07 and 32.93, respectively ; the 
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TABLE 4 
Mean NuMBER or Correct Responses 
AND STANDARD DEVIATIONS FOR 
Groups ON THE ReapinG Task 


Group Mean SD 

WD* 33.00 11.67 
LD 29.53 12.75 
LD-O 32.53 14.59 


* Groups RR and IR combined. 
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learning curves in the reading task for 
the three groups are presented in Fig- 
ure 1. A summary of a Lindquist 
(1956) Type I analysis of variance 
for these data showed no significant 
overall performance difference for the 
groups on the reading task. Compar- 
ing Group LD with Groups WD and 
LD-O separately showed no reliable 
performance differences (F < 1). Per- 


TYPE OF PRETRAINING 


WORD DISCRIMINATION (WO) 
LETTER DISCRIMINATION (LO) 
e-----@ LETTER O'SCRIMINATION- 


ORIENTING INSTRUCTIONS (LO-O} 


4 4. 4 4 4 4 4 


BLOCKS OF TWO TRIALS 


Fic. 1. The effects of visual discrimination pretraining with relevant letters contained in 
words (Group WD: Groups RR and IR combined) vs. relevant letters presented alone 
(Groups LD and LD-O) on correct responses in learning to read a three word list. 


standard deviations, 11.58 and 11.75, 
respectively. A Lindquist (1956) Type 
I analysis of variance showed Groups 
RR and IR did not differ significantly 
in overall performance on the reading 
task. A further comparison between 
the groups on the first trial block also 
showed no reliable differences. 

On the basis of these findings, the 
two word-discrimination pretraining 
groups were again combined and des- 
ignated as Group WD in the reading 
task. The mean number of correct re- 
sponses for Groups WD, LD, and 
LD-O are presented in Table 4. The 


formance differences on the first trial 
block also were not significant. 


Discussion 


If word shape is the relevant visual 
aspect for discriminating among words 
for beginning readers, Group RR pre- 
training should have resulted in trans- 
fer of discrimination learning to the 
reading task and faster learning of the 
list relative to Group IR pretraining. 
On the other hand, if the relevant 
letters alone provide the basis for vis- 
ual discrimination in _pretraining, 
Groups RR and IR pretraining should 
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have resulted in equal transfer of dis- 
crimination learning and no perform- 
ance difference between the groups on 
the reading task. 

The findings based on comparing 
Groups RR and IR on the reading task 
are consistent with the assumption 
that beginning readers discriminate 
among similar length words of differ- 
ent shape on the basis of specific let- 
ter differences. Further investigation 
is necessary to determine the effects of 
at least two other word-stimulus vari- 
ables on visual discrimination learn- 
ing: word-shape differences defined in 
terms of differences in word length and 
location of the relevant letter within 
the stimulus word, i.e., in other posi- 
tions than at the beginning of the 
word. It is possible that placing the 
relevant letter in some middle posi- 
tion in the word would increase the 
difficulty of the letter discrimination. 
This effect could increase the impor- 
tance of shape or length cues as bases 
for discrimination. 

The findings based on comparing 
Groups WD (Groups RR and IR com- 
bined), LD, and LD-O on reading 
task performance require an explana- 
tion to account for the appearance of 
learning differences between the word 
and letter discrimination groups in 
prétraining, and the absence of learn- 
ing differences on the reading task. 
Learning differences in discrimination 
pretraining indicated differential diffi- 
culty in discriminating letters when 
these letters were presented singly as 
compared to being presented in words. 
Absence of learning differences on the 
reading task indicated that the addi- 
tion of irrelevant letters to the pre- 
viously differentiated  single-letter 
stimuli for Groups LD and LD-O did 
not increase the difficulty of discrimi- 
nating these letters. 

To account for the relative difficulty 
of the WD group discrimination learn- 
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ing in pretraining, it is assumed that 
generalization among the nonsense 
words increased as a result of the ad- 
dition of similar stimulus elements— 
the same two letters—to the relevant 
letters in each of the words. Thus to 
discriminate among the words, WD 
group subjects had to acquire a set of 
observing responses related either to 
the location of the relevant letter in 
the words, discriminating the relevant 
letter from the total stimulus complex, 
or a combination of the two types of 
observing responses. 

On the basis of this reasoning, how- 
ever, differences in learning perform- 
ance should have appeared in the 
reading task, at least for the LD 
group. The assumption would be that 
the LD group would need to acquire 
the same orienting responses to the 
relevant letters included in the read- 
ing task words. This additional learn- 
ing would be predicted to interfere 
with the acquisition of the word 
names. The findings did not confirm 
this expectation. 

To account for the absence of a 
WD-LD group performance difference 
on the reading task, two explanations 
are offered. The simplest explanation 
is based on the greater difficulty and 
variability of the reading task. The 
increase in performance variability 
could mask any differential transfer 
of discrimination learning between 
the pretraining groups. A second ex- 
planation requires the assumption 
that exposure to the relevant letters 
in the LD group pretraining resulted 
in an “acquired distinctiveness” of the 
letter stimuli, not detected by the pre- 
training response measure, that trans- 
ferred to the reading task. Transfer 
of this acquired distinctiveness could 
facilitate the acquisition of the ob- 
serving responses necessary to dis- 
criminate the relevant letters included 
in the reading task words—the same 
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observing responses acquired more 
slowly by the WD group in pretrain- 
ing. 

The second explanation suggests the 
hypothesis that although letters in- 
cluded in words are more difficult to 
discriminate than the same letters pre- 
sented singly, this relative difficulty 
in learning to discriminate letters in 
words can be overcome by providing 
visual discrimination training with 
the relevant letters prior to presenting 
them as parts of words. 

Further research is necessary to test 
this hypothesis when the relevant let- 
ters are included in other than the first 
position in the word. 


SUMMARY 


The purpose of this study was to 
determine the stimuli relevant to vis- 
ual discrimination among word forms 
for beginning readers. 

Of three groups of kindergarten chil- 
dren, two received discrimination pre- 
training with words. For one group, 
the pretraining words were con- 
structed so that both shape and letter 
differences among the words were rele- 
vant to discriminating among the 
words in the reading task. For the sec- 
ond group, only the letter differences 
among the pretraining words were 
relevant to discriminating among the 
reading task words. A third group re- 
ceived discrimination pretraining with 
the relevant letters alone. 

There were no significant perform- 
ance differences between the two word 
discrimination groups on either a pre- 
training or reading task, a finding con- 
sistent with the assumption that chil- 
dren discriminate among similar 
length words of different shape on the 
basis of specific letter differences. 

When the letter discrimination pre- 
training group was compared with the 


combined word discrimination pre- 
training groups, there was a perform- 
ance difference favoring the letter dis- 
crimination group in pretraining, but 
no differences on the reading task be- 
tween the groups. These findings sug- 
gested the hypothesis that although 
letters included in words are more dif- 
ficult to discriminate than the same 
letters presented singly, this relative 
difficulty in learning to discriminate 
letters in words can be overcome by 
providing visual discrimination train- 
ing with the relevant letters prior to 
presenting them as parts of words. 
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